Samba 4 Released

Samba 4 has been a long time in coming—and it seems to still be a ways off. Samba is a free implementation of Microsoft's SMB/CIFS protocol that is used in the vast majority of consumer-targeted network storage devices, but the current 3.x versions lack many of the features that enterprise users require (Active Directory support in particular) and Samba 4 is meant to address that shortcoming while also reworking and rewriting much of the code base. The biggest hurdle to a Samba 4 release seems to be integrating the new code with the existing pieces that currently reside in Samba 3. A recent proposal to release the current code "as is"—rather than complete the integration work before a 4.0 release—is under heavy discussion on the samba-technical mailing list.
There is a fair amount of impatience, it seems, for a Samba 4.0 release. In the proposal, Andrew Bartlett notes that vendors would like to build their products atop a stable release, rather than alphas, and that finishing the integration work in 4.1 might allow a final 4.0 release "in about three months time". Samba 4 has been in the works since 2003, with several "technology preview" releases starting in 2006 and the first alpha release in 2007, but a 4.0 final release has so far proved elusive. In his proposal, Bartlett is seeking to route around the hurdles and get a release out there.
Part of the problem with integrating the Active Directory (AD) Domain Controller (DC) work with the existing production Samba 3 code is that there needs to be a clear migration path for users who upgrade. If the existing Samba 3 file server code (often referred to as "smbd") were still shipped as an option, existing users would not need to change anything. Only users that were interested in moving to Samba-based DCs would need to start using bin/samba (which is the name of the Samba 4 server that includes AD and DC functionality).


But, some have always envisioned Samba 4 as a single server process that can handle all of the different roles (file server and AD DC), which necessitates the integration. Others are not so sure that it doesn't make sense to release a Samba 4 that incorporates the new functionality for those who need AD DC support, while leaving other users to continue using the older, working and well-tested code. Part of the problem seems to be that various different sub-groups within the project are taking their own approach, and no one has done a lot of development—or testing—of an integrated server solution.
Those who are currently testing DC setups with the Samba 4 code are using the simpler, single-process "ntvfs" file server code, rather than smbd. For that reason and others, Andrew Tridgell would like to see ntvfs still be available as an option:

For embedded devices the single process mode and ntvfs/ file server code is what we are likely to use for some time, as it uses far fewer resources. For intensive file serving using smbd makes sense but I don't want to lose the ability to run in small environments. 

But that doesn't sit well with Jeremy Allison, partly because of the maintenance burden:

Having a second file server embedded - used only in certain cases and having differing semantics is a [recipe] for disaster, and not a good idea for long term stability of this release. That's a big sticking point for me. I though we'd decided s4 fileserver was smbd code - end of story. The details were how to do the integration.

Beyond just the embedded case, though, Tridgell sees value in keeping ntvfs alive, rather than forcing those users to switch to the smbd-based file server code. It's a matter of what testing has been done, as well as causing fewer disruptions to existing setups:

Apart from the embedded case, it is also the file server we have done all testing of Samba as a DC against so far. Being able to run it by setting a config option is a good thing I think, at the very least for debugging issues with the changeover to the smbd based file server. It also means that existing sites running s4 as a DC are able to continue to run the file server they have been using up to now. 

Tridgell also thinks that ntvfs has a design and structure that should eventually migrate into smbd. But, as he notes, his arguments haven't convinced Allison, so he's started working on a branch that uses the smbd server. That's only one of the problem areas, though. AD DCs handle far more than just file serving, they also perform authentication, DNS lookups, act as printer servers, and more.
But, once again, there are two versions of some of the services floating around. For example, winbind, which handles some authentication and UID/GID lookups has two separate flavors, neither of which currently handles everything needed for an AD DC. Tridgell and Bartlett have been looking into whether it makes sense to have a single winbind server for both worlds, seemingly coming to the conclusion that it doesn't. But Allison, Simo Sorce, and Matthieu Patou see that as another unnecessary split between existing and future functionality. Sorce is particularly unhappy with that direction, saying:

It almost [seems] like we should just fork the 2 projects, after all we reimplement everything differently between the file server components and the DC components with no intention of sharing common code [...]

Part of the complaints about Bartlett's proposal is about positioning. Samba 4 has been envisioned as a complete drop-in replacement for Samba 3. To some, that means integrating the Samba 3 functionality into Samba 4, but for others, it could mean making the Samba 3 pieces available as part of Samba 4. Tridgell and others are in the latter camp, but Allison looks at it this way: "It isn't an integrated product yet, it's just a grab bag of non-integrated features." He goes on to say that it will put OEMs and Linux distributions "in an incredibly difficult position w.r.t. marketing and communications".
Everyone seems to agree that the ultimate goal is to integrate the two code bases, but there is enough clamor for a real release of the AD DC feature that some would like to see an interim release. One idea that seems to be gaining some traction is to do a "Samba AD" release using the Samba 4 code (and possibly including some of Samba 3 server programs). That release would be targeted at folks that want to run a Samba AD DC—as many are already doing using the alpha code—encouraging those who don't need that functionality to stay with Samba 3. As Géza Gémes put it:

Call the Samba 4.0 release Samba-AD (the idea behind the name belongs to Sernet people), and continue to release Samba3 as Samba-FS. This way people would have a suggestion where those are going to be deployable. Of course I DON'T propose the end of the integration efforts. But if the plan is to do a release in the near future that seems a good (certainly not perfect) compromise. Having a Samba release with ability to act as an AD DC is becoming more and more important to many people who have to upgrade their network infrastructure.

Doing something like that would remove much of the pressure that the Samba team is feeling regarding a Samba 4 release. That would allow time to work out the various technical issues with integrating the two Sambas for an eventual Samba 4 release that fulfills the goal of having a single server for handling both the AD DC world and the simpler file-and-print-server world. As Gémes and others said, it's not a perfect solution, but it may well be one that solves most of the current problems.
The underlying issue seems to be that Samba 3 and Samba 4 have been on divergent development paths for some time now. While it was widely recognized that those paths would need to converge at some point, no real plan to do so has come about until now. Meanwhile, users and OEMs have been patiently—or not so patiently—waiting for the new features. It is probably still a ways off before the "real" Samba 4 makes an appearance, but plans are coming together, which is certainly a step in the right direction. Given that some have been using the AD DC feature for some time now, it probably makes sense to find a way to get it into people's hands. Not everyone is convinced of that last part, however, so it remains to be seen which way the project will go.