Cry Babies Cry. Programmers Code.

A schism of sorts is forming in the Debian developer community as one developer has gone on record to formally criticize Debian's decision to adopt systemd in an Open Letter to the Linux World.

Here's what I have to say to Mr. Christopher Barry and others who may agree with his viewpoint.  Accept it.  Systemd is a done deal.  It's here for good reasons whether you realize it or not.  But I hope you eventually grasp why it was written, as it does solve many inherent 'known' problems with aged sysvinit.

And, as expected, a chorus of cry babies has been awoken, like sleeping dogs, taking aim (again) at systemd.

Those who complain in this instance, I am afraid, have a simple agenda.  

They are lazy.

As such and with much creativity they will persist lodging complaints so as to avoid doing some difficult, but not insurmountable, work.  Yes, there are many pain points in addressing merging systemd PID 1 code that are due to its 'middleware' central/critical role, which result in dependency changes and in some cases some major code rewrites that must be done to conform with this new technology standard.

Ah, standard.  There's the rub.  There are legions of arrogant, swaggering Open Source code jockeys who like to strut their stuff by spinning off their 'me too' Distro with their own branding overnight.  Have you taken a count of how many Linux Distributions there are now?

Standardization strengthens the power of Linux.  Distro sprawl does not.

The planned integration of systemd is now officially deployed to Debian Jessie Beta 1. This means that all the 'foot dragger' derivatives must follow suit with doing what is needed to align with this major system design change.

So, while we see some dig in their heels by organizing a boycott, others choose to simply whine, as the din gets louder and louder.  Soon, though, the cry babies will run out of tears, pick up their toys, and go home whilst the real-world Programmers continue to keep their heads down, doing the grunt work with little fanfare and nary a complaint.

Cry Babies cry.  Programmers code.  -- Dietrich

Related articles

• The Debian Init System Deba{te|cle}
• Systemd: Got Choice?
• Developer Dissatisfaction Looms with Systemd

Read More

Two stories critical of systemd taken offline

by Dietrich Schmitz

There are two stories I wrote last year in progression as I waded through a sea of technical information regarding systemd.

The two taken in totality paint a picture that would represent I am against systemd.

In a follow-on story, I continue my technical evaluation of the merits of systemd including cgroups.

My conclusion then and today is that systemd is, on net, a technology advancement which I accept and use today in Fedora 20.

A site has recently been using one of those stories as reference material in a campaign to boycott systemd.  I do not wish to be associated with any such campaign.

Accordingly, I have taken off-line those two stories which are now inconsistent with my viewpoint.  

People can change their minds about things, and often do, as I have done.

All hail Lennart Poettering.  -- Dietrich

Read More

cgroups: A BIG WIN for Systemd

by Dietrich Schmitz

Oh dear.  What have I done?  :/

I am afraid I have stirred up a hornets nest.  With several related articles under my belt regarding systemd, I continue to pursue this subject matter because of its great importance.

As an advocate, it is important to be critical.  It is also important to turn things over and see things in as objective thinking as is possible, looking for the merit. (Image credit: Red Hat)

Today, I focus on some of the merit that has many so enthusiastic about systemd.

In particular, cgroups, a new technology in the Linux Kernel has been integrated into systemd.

What Matters to System Administrators and Users?

The truth is, most users won't even see or give a hoot about cgroups or systemd, nor will they care anything about their Distro other than how they use it for every day things.  And that is to be expected.

But to the Linux System Administrator who must keep servers numbering in the hundreds or more stable and running without interruption, the devil is in the details and necessarily they must know and understand systemd to accomplish routine configuration changes and to respond to situations that require attention such as when a daemon service becomes unresponsive and must be bounced (stop/start).  These kinds of things are what System Administrators do and the process of learning new technology is a necessary ingredient to staying on the cutting edge and maintaining proficiency.

So, I've spent considerable time poking at systemd, yes?  This time I look at why, just why, maybe there is merit in employing systemd.  Now, many of you are ready to take issue with me, but please remember my role here.  Advocacy is 'for' and 'against' so I've been most often against certain things.  Putting aside how systemd was designed (merging udev) and what that did to the larger Distro community, cgroups does indeed offer something quite good.

The Return of the Zombies

You see it often.  A 'Zombie' child process or an unresponsive daemon service requires your attention.  It is a source of continual maintenance when a service goes silent or one of its child processes does and the whole daemon service shuts down randomly with no explanation.  System Administrators say: "It just happens now and then, that's why we run Nagios to stay on top of it."

That's fine and tools like Big Brother and Nagios are what is needed in this context.  But does systemd offer a finer degree of control given that it has croups embedded?

The answer is yes.

cgroups and Why They are Important

To the extent that Red Hat has a vested interest in the furtherance of systemd and plan to deploy such technology in their next iteration of Red Hat Enterprise Linux 7, much, if not all, of the work being done thus far on systemd has been undertaken by one of their employees +Lennart Poettering.  To his credit, Lennart has been around the corner, so to speak, and has a track record for writing many complicated projects, including PulseAudio.   

So, where can one find more about cgroups?  It just so happens that Red Hat have been doing their own homework, per usual, and have many on-line resource materials that help educate the System Administrator, including some good information about cgroups here.

In the course of discussion in the comment section of my website, one article, stirred the hearts of many to come forward and respond.  One individual has done a good job of giving good constructive replies and demonstrates a better understanding of systemd than perhaps any other respondent I have read.  In my story: Systemd: An Accident Waiting to Happen, Peter Dolding writes:

Shell scripts don't trace processes. This is the problem scripts and items like it work as long as everything is well behaved.

As soon as you get malfuncting service. Solaris zones or linux cgroups around it keeping track of every process it starts. Becomes a good send. 

The problem for over 30 years the same thing:

1) init starts a service process.
2) The service process starts some other sub process it needs.
3) The service process gets terminated to restart
4) The sub process stays running.
5) service started falls over in a heap because sub process is still alive. 

That 1 to 5 (rundown) to your normal init systems includes (Ubuntu's) upstart. 

Solaris and systemd due to usage of zones and cgroups (respectively): 

1) init starts a service process wrapped.

2) The service process starts some other sub process it needs.

3) The service process gets terminated to restart

4) The sub process stays running.
5) Init system checks if anything is left in the cgroup or zone that should have been terminated.

6) service restarted stable.

This is why the old system is broken Dietrich.

Is systemd the right fix needs to be the question--not putting your head in sand and pretending problem does not exist. 

Dietrich the other big thing you have missed is that udev is going into the Linux kernel. As a kernel feature. BSD and other operating systems really have to catch up. 

There are major performance issues running udev userspace. There is also security issues runnign udev userspace. There will be no need for systemd to look after udev long term. Because udev will not exist.

Instead kernel udev will be used.

It's the same kicking and screaming over moving video card memory management into kernel space.

This is nicely put.  I have yet to see anyone explain it better and in such concise terms.  Well done Peter Dolding.

Conclusion

If more people will step back from the issue and wrap their minds around the above, I think it will help them to grasp what is happening to Linux in the long term.  Does breaking compatibility with things related to BSD matter?  I am not sure on that.  Debian chose to make systemd an opt-in choice for their just released Debian 7 Wheezy project, specifically because it does break GNU/kFreeBSD.

I want to believe their is merit in systemd and so, in this round, I am looking at the glass 'half-full' as opposed to being 'half-empty'.  

I pledge to l continue, in investigatory fashion, to examine all things which effect Linux, turning them over to see them from different perspectives.  That is necessary to being a good advocate.

With that, I hope this post has done some good.  What are your thoughts?

-- Dietrich

Related articles

• Systemd: An Accident Waiting to Happen
• Slackware: Is Systemd Inevitable?
• Systemd: The New PulseAudio
• Gentoo Team Isolates Udev from Systemd (eudev)

Read More

Slackware: Is Systemd Inevitable?

by Dietrich Schmitz

I recently wrote a story Systemd: The New Pulse Audio as regards the work undertaken by Red Hat and one employee +Lennart Poettering to improve Linux with a middleware daemon called systemd.

Much press has been given to the story surrounding systemd putting the work into question. Still, the work progresses and continues to move forward and many Linux Distros have chosen to adopt it as others opt to take a 'wait and see' approach.

So, far, the oldest Linux Distro Slackware has avoided the issue and the most recent version, Slackware 14.0 released last fall by its Founder +Patrick Volkerding, is doing just fine without it.

I came across an interview done by LinuxQuestions.org with Patrick and share here a passage in his interview which deals directly with the issue of systemd.  Patrick writes:

"...Concerning systemd, I do like the idea of a faster boot time (obviously), but I also like controlling the startup of the system with shell scripts that are readable, and I'm guessing that's what most Slackware users prefer too. I don't spend all day rebooting my machine, and having looked at systemd config files it seems to me a very foreign way of controlling a system to me, and attempting to control services, sockets, devices, mounts, etc., all within one daemon flies in the face of the UNIX concept of doing one thing and doing it well. To the typical end user, if this results in a faster boot then mission accomplished. With udev being phased out in favor of systemd performing those tasks we'll have to make the decision at some point between whether we want to try to maintain udev ourselves, have systemd replace just udev's functions, or if we want the whole kit and caboodle...."

Given the foregoing and given that historically Gnome was removed from Slackware, it would seem that Patrick is being careful not to tip his hand one way or another as to whether he will 'yield to pressure' and become systemd-compliant.

I speculate (Patrick's development is kept private so nobody get's access to his decision making until he says so), that Patrick's plan's for the next revision may well include software design changes that successfully keep systemd out of his implementation.

If that is the case, it would be virtuous if other Community Developers fell in lock-step and provided him assistance in such a worthy effort.

Personally, I can understand the contention surrounding systemd's invasiveness and how it violates the UNIX concept of "doing one thing and doing it well".

Specifically, having the initialization process loosely coupled to a series of run level cascading shell scripts has served Linux well for many years and maybe the concern that work being done on systemd is too ambitious insofar as bringing too many processes under control of one daemon service, systemd is warranted.  Add that the logging of activities taken by systemd are no longer human readable, but journalized in a binary format and you have a recipe for programmer revolt.

It was most recently the genesis of a decision by Fuduntu to close its doors as the volume of programming required to merge needed systemd changes was simply too large an undertaking for its meager staff to handle.

Will Slackware be the next Distro to fall in line and become systemd-compliant?  Stay tuned for developments.

-- Dietrich

Related articles

• Confused by FuSE
• Developer Dissatisfaction Looms with Systemd
• The Other Shoe Drops: Founder Announces Retirement, Fuduntu End of Life
• The Road Not Well Traveled: Two Developers Think Outside the Box
• Systemd: The New PulseAudio
• Have Linux Distros Gotten Too Tubby?

Read More

Developer Dissatisfaction Looms with Systemd

by Dietrich Schmitz

Things seem to be moving along swimmingly with ongoing development surrounding systemd, at least as far as Lead Developer Lennart Poettering is concerned who has outlined the plans for moving the project forward with hopefully full upstream community participation in lock step.

Some of the dissenters have been quite vocal about having systemd 'forced' on them.  Countering that sentiment is the prevailing argument that developers don't have to use systemd.  +Fabian Scherschel opened a talkback thread regarding my Linux Advocates story Systemd: The New PulseAudio in a public Google Plus post made yesterday with this comment:

There is no controversy. Unlike PA, it was adopted when it was ready to be adopted and everybody seems to like it very much. With the exception of the Ubuntu people but that seems to be based on NIH alone. This is a completely different situation then PA.

Fuduntu Founder +Andrew Wyatt returned fire with this volley:

ConsoleKit + UDev + Syslog + DBus + Polkit + Sysinit + this + that.   RedHat Enterprise Systemd is the best product we've ever been force fed.  We are facing being forced to integrate it at Fuduntu because it's replacing so many core tools now that it's impossible to continue the project without it.

It seems fairly understandable that smaller Distro Teams are indeed resource-strapped and so cannot muster the resources to refactor their programs to by-pass the new systemd project's major structural changes.  The volume of work necessary to avoid systemd will effectively result in forcing the Developers to do just what Mr. Wyatt describes--be forced to use systemd.  I would add that the other alternative would be to 'end of life' a distribution.

That would be most unfortunate if that were to happen and the possibility of many fledgling Distros not being able to accommodate systemd to their own demise seems to be a distinct possibility.

Will a Linux Distribution 'consolidation' occur?  That remains to be seen and Linux Advocates will continue to follow this story closely.

The matter of personalities also, unfortunately, has caused some undue friction as systemd Lead Developer Lennart Poettering, also developer of PulseAudio, has a penchant for freely speaking his mind and this has been a source of conflict that might have been completely avoided had he exercised a large degree of self restraint, tact and diplomacy in making public statements.

In one such statement, Slashdot.org ran this story:

"In an interview with LinuxFr.org, Lennart Poettering speaks freely about his creations, PulseAudio, Avahi and systemd among other things. Naturally, what has stirred up most of the discussions online is Lennart's opinions on BSD. Following the recent proposal to make Gnome a Linux-exclusive desktop, Lennart explains that he thinks BSD support is holding back a lot of Free Software development. He says this while also taking a stab at Debian kFreeBSD: 'Debian kFreeBSD is a toy OS, people really shouldn't misunderstand that.'"

At this juncture, it's clear that systemd will not support kFreeBSD.  That appears to have not changed since the above story ran in 2011.

Debian's next operating system revision, is code named "Wheezy" (Debian 7.0).  An official release date has not yet been announced but the NewInWheezy page does confirm that systemd is included as an installable 'option'.

So, the tension is palpable and right under the surface is brewing dissatisfaction among the rank and file developers who must either deal a huge undertaking of refactoring code, packaging, etc., or cave in and simply implement systemd or worse simply perish.

The writing is on the wall.  Implement systemd or you are on your own, like it or not.

-- Dietrich
[Edit: 4/15/2013 9:00am GMT-5 This story just in confirms what I feared.  Fuduntu has been end of life'd.]

Related articles

• Systemd: The New PulseAudio
• Systemd 199 Has Its Own D-Bus Client Library
• Of Standards, Trends, and Shiny Things
• Systemd 198 Brings "Many Big Changes"

Read More