Slackware, Unix, PLASMA and Nethack: A discussion-based retrospective of the 1980s to the 2010s - Pt II
Earlier this month I conducted a discussion with a close companion and programmer who I will call M. M has been programming since the 1980s and has been using a host of different languages and operating systems since. Going from CP/M and Unix-based systems at work and home in the 1980s to Slackware in the 1990s and the 2000s, this discussion maps out his experiences and opinions on the world of free and proprietary software, parts of which will hopefully be of some interest. He is not, as far as I am aware, registered or active at LQ.
N.B.: apologies for any grammatical/syntactical errors, or any inconsistencies in the timeline. I have tried to remain as faithful as possible to the original audio.
Due to character constraints, this entry will be published in two parts.
+-----------+
Part I is here.
+-----------+
Previous entry - Unix, Gaming, Santa Cruz Xenix and Slackware.
This entry - PLASMA, systemd, Nethack, security and the Unix philosophy.
PLASMA
L: How long did you stay with [Slackware] for [in the 2000s]? Because you left it at some point.
M: KDE. KDE killed it for me.
L: What version KDE?
M: I can’t remember if it was 4, I think it was 4. It was the one with PLASMA. Remember I was running this in a virtual machine on OSX. I loved KDE, I was in there I think it was from beta 2 or beta 3, so from very early on I was running KDE and I loved it and I started to get really into it and I was doing funky stuff with it because it was very good. Suddenly there was a new version, I can’t remember whether it was version 4 or version 5 and it was clearly trying to look like OSX. I remember the settings screen, it started to look like OSX, it was as annoying as OSX, it was as irritating as OSX, it was as rubbish as OSX and I thought, “I may as well run bloody OSX”. So I did. I got rid of my virtual machine and Linux and started running OSX. Why not? They were trying to make it look the same but OSX did it slightly better so what was the point? I just thought it was awful. Absolutely awful. From something which I thought had been fantastic. I thought it was a real step backwards, it didn’t even work very well when it came out either, which you can kind of expect but KDE had always been rock solid before that, and fantastic, and it had DCOP and all these little things you could use for communicating between applications. And they changed the whole lot then, fair enough, I think that happens a lot in the open source community, but this new version redoes the code, like anybody cares. I understand that it’s probably beneficial but the users don’t give a damn what the code is like as long as it actually works.
L: That must be a big concern.
M: Oh it still happens now. GIMP did it as well, a great big code rewrite which it probably needed but it’s not very exciting for anybody. If you want to use GIMP, if you want to actually use it rather than play with it because it’s great to have a tool that does that, if you want to actually use it professionally or use it seriously, you don’t care. You just want it to do the stuff. I do get that a code rewrite is important but KDE and GNOME actually are constantly talking about toolkits and libraries, I don’t care. But when KDE4 came out, it was ugly, it didn’t work very well, I’ve never understood why people want things to look like Windows or OSX. Why were they doing that when they had their own thing? Take Enlightenment as a desktop. Now that wasn’t trying to look like Windows. That was not trying to look like OSX. That never went very far but that was really trying to do something different. There was another desktop, a 3D desktop, I can’t remember what it was called – totally unusable but it looked really cool.
L: Why did you not consider running Xfce or something and just running with that?
M: My machine was basically looking like the system which was the host operating system for it. The host operating system was going to be much better. I don’t like OSX very much, I don’t think it’s very consistent, it is at least half Unix, things you plug in work first go. Maybe I just got older. In the old days I would spend days trying to get things to work on Linux and enjoy it, I remember taking two days off work sick trying to get ImageMagick, I think, some old tool which may or may not be around, to write text over jpegs. I think the problem was getting it to compile a jpeg library or something. I spent two days trying to get that to work and I was seriously excited when it did work. Maybe I was just getting less interested in banging my head against a wall when my Macbook Pro just did it, I didn’t have to bang my head against a wall. I have OSX now and I still don’t like it.
L: There’s one other person who’s expressed similar sentiments about their frustration with what happened with Linux and that was this forum guy called dasein, a German word, a controversial chap and a lot of people didn’t like him, but he and I were very good friends before he passed away. Your exact words in an email to me a few years ago were that you’d decided that all operating systems were crap and made you want to chew your own eyeballs out. He said that his sentiments were more or less exactly the same and if Windows 7 were going to be supported longer-term out of Redmond then he would ditch Debian – because he was using Debian but he was anti systemd, hugely anti systemd – and he would run with Windows. So I wonder if that’s a common symptom after spending so long in Unix and seeing how things have gone and thinking “screw it” and going for the easiest route.
M: Maybe yes, but when we started having Unix-like systems at home you didn’t expect GIMP to work, Blender to work, we didn’t expect these things to work and they weren’t available in Windows and they weren’t available in Macintosh but there is vast amount of tools and applications we have now, but then we couldn’t get frustrated that they didn’t work -
L: It was nice to have them.
M: Yeah, and when I first found GIMP I thought it was a fantastic piece of software and it worked and there were big fights about it - libraries, GNOME libraries - I always battled between that and the other one which KDE was using, I can’t remember the name of it now, which is odd because I went for a job there.
L: Did you get it?
M: Yes, I did actually. Actually it wasn’t a job there it was a job at Trolltech - the people who worked at Trolltech had set up a new company - and I was offered a job there, which I accepted. All those fights and arguments drive me nuts, but we have very complex software now, very capable software, very good software and we use it for all sorts of things so maybe now the operating systems do need more complexity and annoyance, or maybe it’s simply that they are trying to make them accessible for everybody... but none of them are. And they shouldn’t be, why should they be? You don’t expect everybody to fix their cars, in fact people are less and less able to fix their cars, like their phones – [picking up his phone] you can’t fix that – you have to take it to experts, I don’t see any problem with experts taking on very technical things.
L: What are your thoughts on distributions like Ubuntu and Mint then?
M: I just never cared. There were lots of different distributions that came out and Debian had some philosophy behind it -
L: They have a social contract.
M: And I’m not interested in that, I just wanted to run the operating system and for it to be as elegant and beautiful as Unix is. When Red Hat became commercial and IBM got involved with them and everyone got involved with them, it was great but they changed it, they added all sorts of crap like systemd. It’s not the first time they’ve done something that was different to what everybody else wanted. There was another big change around sound or something, but it all became a bit fragmented and overly-complicated because underneath all this complication is something that still works perfectly and simply.
systemd
L: I’ve got to mention it, what was it about systemd that annoyed you so much when you first came across it?
M: I’ve never actually used systemd myself so you have to bear in mind that if I did I may think it’s absolutely brilliant and fantastic and like, why didn’t they think of this before? But as I mentioned to you in the car the other day just before hearing about systemd I had been working on a Solaris box, and I think it was the utmp file that was corrupted or one of the user logging files, and Solaris’ dependency boot system refused to start sshd because utmp wasn’t available. It didn’t mount it for some reason, I can’t remember. The dependency software didn’t load sshd so I couldn’t log in to the box purely because utmp was not there and you don’t need utmp to log into the box. And the solution, if you don’t have utmp, is to go mkdir and create a new one! But I couldn’t log into the box. And at the particular organisation you couldn’t just log into a production system, you had to go through all sorts of security gates and have two people with keys to get into a box in order to get the bloody thing to work but I couldn’t log into the ssh. So I didn’t have a very good feeling about startup dependencies to start with.
L: So you thought that trouble was on the way.
M: Well, I just experienced a problem with dependencies that pissed me off. Then I started reading about systemd and I thought, well, what problem are you trying to solve here? And I remember after many many years of seeing things not behaving as they should be and going back through the steps of how that system had booted and going through all the configuration files and I realised I couldn’t do that with systemd. I’m sure you can get the stuff out, the information, but it’s not obvious how to do it but the way the machine boots [with e.g. SysV] is simple and has been around for a long time and it works so well. You can get into dependency problems when something has not started but you tend to know while you’re testing it, not in production because something has stopped behaving correctly. It could happen, I admit that it could, but for me it’s complexity over simplicity and I can’t see that the complexity is giving me anything of benefit.
L: In my experience which is scant in comparison to yours, systemd is less stable. I run four Slackware systems and I’ve never had a failed boot, not once, on any of them.
M: I’ve never had a Linux boot fail.
L: The only time there’s been a failed boot is because I did something wrong.
M: Yes, I have done the same thing myself. I remember my biggest one was at a customer’s, a two day contract, with some government. I can’t remember why but I wanted an enormous sleep running in the background when you boot the machine. There was a bit of a panic, it’s getting late in the evening on the first day and I only had one more day, I booted the machine, it hung, and that was a nightmare. I had no idea why. I remember going home that night really nervous because I had to – had to – get the job finished the next day and I had a machine which had been working and I’d now basically broken it. I remember lying in bed thinking, “oh shit, I bet I left the ampersand off the end of that sleep”. I thought, well if I have, it probably will have booted in the morning... and I got there and it had. That’s probably the most scary moment that I’ve had and I’ve done some stupid things. There was a national computer dealership, I can’t remember their name, and I was working there in their computer room which was small, it was full of IBM computers and me and the other guy, we used to go in there to smoke. I remember fiddling about with a chair and banging my knee against one of the disk drives and it powered off and it shut down the whole country for that dealership.
L: It does make you wonder how many of these unscheduled outages happen at big companies because someone has tripped over an Ethernet cable.
M: That happens a lot, that does happen a lot. I remember hearing a story about someone shutting down an Air Traffic Control system, I think he’d typed ‘shutdown’ instead of ‘man shutdown’ and he got the sack for that. Something like that, something really stupid, something that anyone could do.
Nethack
L: We should wrap up soon because we’re coming up to 75 minutes...
M: Before we do, we do have to mention the most important game, which hasn’t been mentioned yet which has been with me through my earliest career, in fact before I started in computers.
L: Repton!
M: How do you remember that game?
L: I don’t know.
M: No, it’s Nethack.
L: I’ve heard of it, I haven’t played it.
M: Still the most incredible game ever made. A roguelike game, and they’re called roguelike because of the original Rogue. An ASCII adventure game. If you ever get the chance, check out Nethack.
L: What’s so good about it?
M: It is beautiful, it is elegant, it is insanely difficult to complete. I’ve never completed it, I’ve played it for forty years, on and off, I’ve never completed it. It is a Unix game and it’s just beautiful.
L: When you say ‘elegant’ - this is a word I hear coming up – what are you talking about? Elegance of operability, elegance of coding?
M: It’s been a long time since I’ve been involved in all this but I think there was a phrase that people who play Nethack used to use which is “developers of Nethack think of everything”. It’s a very free game, it’s an ASCII character walking around a maze, basically, and there’s lots and lots of little items, magic things, potions, things like that, but you mix them together in a way that nobody’s ever thought of doing that the developers have, and it will react in a way that makes you think “wow, they’ve thought of everything”. It’s an amazing game.
‘Unix philosophy’
L: Looking back, what was your favourite era of computing, what has been the most exciting and interesting for you? Not necessarily what the ‘golden age’ is because that’s subjective – and though this is also subjective it’s more relevant to this discussion.
M: That’s actually quite an interesting question because there have been very specific eras in my computing and they’ve all been very different. I mean, at Baker Street it was great fun turning up at 3am to some accountant, someone who’s got a problem with his computer, in your Land Rover, beaten up old Land Rover, and he’s got a bottle of Chablis in there while you’re trying to fix his computer, it was great. That’s one great thing. Writing system software for national ID systems and fraud detection systems just recently, well fairly recently: it’s fun and interesting but a lot of my answer would depend on who I was at the time as well. It’s all been interesting in different ways and now, right now, when I’m learning CAD software, Fusion 360 and I’m learning Blender for 3D modelling. It’s not operating system level but it’s a good fun thing to be doing. It’s all been different, there’ve been no dull moments.
L: There’s no time which you think it better than another time?
M: No, different, not better. You can have that same feeling as I did when I first got that crack to appear of my space-trading warfare game - multi-user - written in Sculptor report generator, the same feeling of making that work as making a cloth wrap around a solid object in Blender now.
L: That’s kind of an intellectual satisfaction, in the way of learning and accomplishment.
M: It’s problem-solving and seeing the problem fixed. That’s what Linux has always been able to do, and Slackware particularly, because it lays itself open to you and if you’ve got the time and the effort to learn what it’s showing you, you have all the tools you need to do everything and you can find problems and fix problems and solve problems using those tools like you have with Perl or with Python or with C or with Blender or with Dungeon Master. Certainly Nethack - “two dead humunculi” - not “one dead humunculus”, you have one dead humunculus, two dead humunculi!
L: Which systems are you running now at your workplace?
M: Red Hat. We always have some Solaris boxes and one or two Windows boxes but mostly Linux, it’s a Linux shop. We have some SUSE too.
L: So where do you think it’s all going to go? Do you think the user base is going to get increasingly diluted by more people who don’t know and don’t care about their systems? Is it a generational thing? Fewer and fewer people caring about systemd and whether it’s in their computer or not?
M: New people won’t care about systemd because they’ll grow up with it. I think it’s a shame because Unix has been around since the ‘60s in one way or another. It shouldn’t be that something has been around for that long, it has changed of course, but that elegant small tools Unix philosophy, if it’s lost - so many things get lost over time - it would be a shame. Things will still go on. Ultimately I can’t see Linux on the desktop going much further than it already is, but Linux everywhere else and in everything else, yes. My car’s got Linux in it, my phone’s got Linux in it.
L: Your microwave?
M: Can’t be, because it annoys me too much. Everything’s got Linux running in it, or the root of Linux, like Android, its roots are in Linux, iPhone. It’s everywhere, the long battle that’s been around since I’ve been in the industry was between Windows and Unix, and Unix was always looking to be the loser but it turns out it’s very much the winner.
L: When you try to convince a lot of younger users about systemd and why it could be considered a bad thing, one of the epithets that often gets used is the ‘Unix philosophy’. It’s said that a lot of people don’t really understand what it means, so what does it mean to you?
M: It’s just simplicity. Unix is simple and it does what it says it’s going to do and you have total, total control. It never does anything insane or crazy or unexpected. When they say the ‘philosophy’ they mean there’s lots of small tools that do one thing and they do it well and you build them all together to make big tools like you make a house out of bricks, and that’s what people mean by the ‘philosophy’. For me the ‘philosophy’ is possibly more about the elegance of it all, the design of how things are put together. It’s not just systemd that annoys me, I get annoyed by the echo command being an internal bash function rather than an external command - I get why it’s there but for me it loses something for it - and I’ve noticed now that you can tar-*. When you use an asterisk or any kind of similar expression, the shell expands that into the current directory, so if it comes up to a lot of file names they get appended to the end of the command and that’s why they work. But it doesn’t work with 'find', I think it was with find where it doesn’t work, because people don’t remember that and they just think it means all files that match ‘this’ - I’m not explaining it very well because I don’t remember it that well - but now I notice that find has been changed, so that it actually does what you’d expect rather than what it should do, which is break. There are lots of little things like that, things which have been ‘fixed’, tiny little things which have been ‘fixed’ but only because people kept making mistakes whereas what they should have done, in my view, is not make the mistakes.
L: I think what you’re talking about, when you talk about the Unix philosophy, is an ethos. And distributions have ethoses behind them - Debian was all about free software, Ubuntu was about making things accessible and Slackware was about making things as Unix-like as possible - there’s a genuine intention behind that and that’s one of the things that makes it - I don’t want to say a work of passion necessarily because that makes it sound ephemeral - but there’s a genuineness behind it which comes across in how it works and how stable it is. Maybe that’s what’s behind it, that intention behind the philosophy.
M: There are philosophies in different distributions as you said, and that’s why I never got interested in any of them, I was not interested in any of the philosophies of the distributions, I didn’t care, I really didn’t. If I’m honest I didn’t care about the GPL - I’m glad for it, I didn’t care about it at the time - I didn’t care what Richard Stallman was saying that we needed, that we shouldn’t do this, we should do that, we should call it this, we shouldn’t call it that, I admired the guy, like I admired all those people but the only ‘philosophy’ that I personally could identify with was Slackware trying to do things right. That’s the philosophy - for me it never looked like it was trying to stay as close to Unix as possible because Unix is all but disappearing - it was more that it was trying to do it right and remember what was good in Unix, and AU/X started to go a bit funny and move away from what Unix was, but I think Slackware’s always kept it, kept what Unix was always supposed to be. I like that, and if I installed Linux again on one of my machines which, as I said to you, I nearly did the other day, it would be Slackware.
Security
L: With regard to security, people say they’ve never seen a Spectre or Meltdown attack in the wild. People are patching their machines like crazy and using microcode but I hear that there’ve been very few attacks.
M: What I’m going to say is very politically incorrect but I have never suffered from a security attack - well, I may have had an attack but I’ve never suffered a security incident. I’ve seen security incidents and I work in an area currently which is very sensitive to security but I’ve never ever seen anything. We run machines that have uptimes of 1,500 days that we don’t dare reboot anymore. Is that a good thing? No, it’s certainly not compliant with some of the auditory requirements that we’re supposed to be following and those machines are special-case machines, if you manage to penetrate the security in front of those machines - to get to them - in my view you deserve to have access! Because they’re not easy machines to get to, we can’t even get to them within the organisation. So it’s very difficult to say. My personal gut feeling is it’s too much of a religion and I have seen problems introduced by patching, I’ve seen systems break because of patching, I’ve seen new security holes arise because of patching. I’ve been to security conferences where they show you how easy it is to break into machines and no amount of patching is going to fix that and no amount of security scanning software is going to solve the problem.
If you can patch, and you can patch regularly and you can patch reliably and you can patch automatically then you may as well do it, as long as you have a good testing regime in there so you can test the stuff first. I personally, on my phone and on my computers, I make sure I’m not at the latest patch level.
L: You’re not?
M: No, I don’t want to be at the latest patch level.
L: Why not?
M: It’s nearly always broken in one way or another. I don’t like it, I hate it, I hate upgrading stuff on my phone, it’s always worse than it used to be.
L: How much risk is the average home computer user at?
M: No doubt there’s a risk of becoming part of a botnet. Of course there’s a risk, so what Windows do with their occasional patch releases is probably a good thing, but religiously patching with every single patch that comes out? I don’t think it’s worth it. I actually think it’s damaging. If you can afford it and you’ve got the time, it’s not going to do you any harm, well, it’s not going to do you much harm.
L: It’s kind of an “if it ain’t broke, don’t fix it” kind of thing?
M: I’m certainly a big believer in "if it ain’t broke, don’t fix it".
L: So having a 1,500 hour [uptime] -
M: Day.
L: Day, sorry.
M: I’m not proud of it.
L: You should be!
M: Actually in some ways I am proud of it, but the fact is that this machine has been running a national identification system and we have other machines with that kind of uptime and no security breaches ever. There was a security alert on some library, I think it might have been TLS or SSL, some kind of encryption algorithm, I can’t remember, but it was some library, an enormous hole had been found in it and everybody needed to panic about it, the whole organisation and the whole of all the organisations were jumping up and down needing to patch this hole in this library but ours were so old that we weren’t affected by it. It was quite pleasant at the time.
L: People are religiously patching and worrying about the next incremental kernel update or the microcode, they’re obsessed by it.
M: Being on the latest version is never a good idea. Oracle release patches to their database, I think four times a year. Solaris used to release patches four times a year and HP-UX four times a year. There are more people desperately trying to get into these machines now but you never stayed on the latest version so you were always six months behind at best. Because the latest version will introduce bugs. At the place I work now - I think it was SUSE which changed the way that su worked. It took me a while to find out that it was su but we patched it - we had automated patching, if we want to patch we don’t go and patch all our machines individually - and we rolled out the latest versions of patches and some obscure script that somebody had written just broke, and it was an important script, it ran once every six months and it broke. And the reason why it broke was that the guy had written the script properly, really carefully and really well, redirecting standard outputs, standard error, and it didn’t work with su because there was a new feature with su and you had to put some incredibly annoying and fashionably long flag on the command line to get it to behave in the same way that it used to. It’s not a huge thing, I suppose, but we had a production failure because of a patch.
We had a change control system written in Perl at one place, some guy patched another machine which patched Perl and it broke the change management system because it changed the way it handled databases. And the change management system failed. It broke because he had patched the system. It’s not a religious thing to do. I think it’s something to do with care, even with Oracle. The best practice with Oracle is to look through every patch first before you apply it.
L: Slackware -current is a testbed for the next stable. It’s not like the very very latest things get dropped into it, sometimes they get dropped into /testing and then they end up going into -current but -current is for the users to report any bugs or if they need it because it’s got more recent hardware support. But the stable version only ends up having security updates.
M: Yes, don’t keep fixing it until you break it. It’s a difficult thing to say. People do get attacked and machines do get taken over and they do become part of botnets which attack other machines, it does happen. It’s just never happened to me.
L: Have any of your colleagues had serious issues?
M: The only serious security issue I’ve ever known was an enormous security issue caused by the customer doing something really stupid.
L: Can you say anything about what it was?
M: A customer sent us his private key through email. That is a serious security incident.
L: Did anything happen as a result?
M: Yes, we had to create certificates, we had to basically go through a key ceremony again.
L: So some of the most serious computer security issues are PEBKAC?
M: Yes, that is the only time I’ve ever been involved in something that’s been called a security incident or that has been confirmed as a security incident. A firewall, or patching, would not have fixed that customer being stupid.
L: Or PICNIC - Problem In Chair, Not In Computer. OK, we'll probably do more at another point but I think that’s enough for now. Thank you very much.
M: No problem.
3rd Oct 2019. This conversation was fuelled by a respectable quantity of beer, wine, e-cigarettes and dog noises.
N.B.: apologies for any grammatical/syntactical errors, or any inconsistencies in the timeline. I have tried to remain as faithful as possible to the original audio.
Due to character constraints, this entry will be published in two parts.
+-----------+
Part I is here.
+-----------+
Previous entry - Unix, Gaming, Santa Cruz Xenix and Slackware.
This entry - PLASMA, systemd, Nethack, security and the Unix philosophy.
PLASMA
L: How long did you stay with [Slackware] for [in the 2000s]? Because you left it at some point.
M: KDE. KDE killed it for me.
L: What version KDE?
M: I can’t remember if it was 4, I think it was 4. It was the one with PLASMA. Remember I was running this in a virtual machine on OSX. I loved KDE, I was in there I think it was from beta 2 or beta 3, so from very early on I was running KDE and I loved it and I started to get really into it and I was doing funky stuff with it because it was very good. Suddenly there was a new version, I can’t remember whether it was version 4 or version 5 and it was clearly trying to look like OSX. I remember the settings screen, it started to look like OSX, it was as annoying as OSX, it was as irritating as OSX, it was as rubbish as OSX and I thought, “I may as well run bloody OSX”. So I did. I got rid of my virtual machine and Linux and started running OSX. Why not? They were trying to make it look the same but OSX did it slightly better so what was the point? I just thought it was awful. Absolutely awful. From something which I thought had been fantastic. I thought it was a real step backwards, it didn’t even work very well when it came out either, which you can kind of expect but KDE had always been rock solid before that, and fantastic, and it had DCOP and all these little things you could use for communicating between applications. And they changed the whole lot then, fair enough, I think that happens a lot in the open source community, but this new version redoes the code, like anybody cares. I understand that it’s probably beneficial but the users don’t give a damn what the code is like as long as it actually works.
L: That must be a big concern.
M: Oh it still happens now. GIMP did it as well, a great big code rewrite which it probably needed but it’s not very exciting for anybody. If you want to use GIMP, if you want to actually use it rather than play with it because it’s great to have a tool that does that, if you want to actually use it professionally or use it seriously, you don’t care. You just want it to do the stuff. I do get that a code rewrite is important but KDE and GNOME actually are constantly talking about toolkits and libraries, I don’t care. But when KDE4 came out, it was ugly, it didn’t work very well, I’ve never understood why people want things to look like Windows or OSX. Why were they doing that when they had their own thing? Take Enlightenment as a desktop. Now that wasn’t trying to look like Windows. That was not trying to look like OSX. That never went very far but that was really trying to do something different. There was another desktop, a 3D desktop, I can’t remember what it was called – totally unusable but it looked really cool.
L: Why did you not consider running Xfce or something and just running with that?
M: My machine was basically looking like the system which was the host operating system for it. The host operating system was going to be much better. I don’t like OSX very much, I don’t think it’s very consistent, it is at least half Unix, things you plug in work first go. Maybe I just got older. In the old days I would spend days trying to get things to work on Linux and enjoy it, I remember taking two days off work sick trying to get ImageMagick, I think, some old tool which may or may not be around, to write text over jpegs. I think the problem was getting it to compile a jpeg library or something. I spent two days trying to get that to work and I was seriously excited when it did work. Maybe I was just getting less interested in banging my head against a wall when my Macbook Pro just did it, I didn’t have to bang my head against a wall. I have OSX now and I still don’t like it.
L: There’s one other person who’s expressed similar sentiments about their frustration with what happened with Linux and that was this forum guy called dasein, a German word, a controversial chap and a lot of people didn’t like him, but he and I were very good friends before he passed away. Your exact words in an email to me a few years ago were that you’d decided that all operating systems were crap and made you want to chew your own eyeballs out. He said that his sentiments were more or less exactly the same and if Windows 7 were going to be supported longer-term out of Redmond then he would ditch Debian – because he was using Debian but he was anti systemd, hugely anti systemd – and he would run with Windows. So I wonder if that’s a common symptom after spending so long in Unix and seeing how things have gone and thinking “screw it” and going for the easiest route.
M: Maybe yes, but when we started having Unix-like systems at home you didn’t expect GIMP to work, Blender to work, we didn’t expect these things to work and they weren’t available in Windows and they weren’t available in Macintosh but there is vast amount of tools and applications we have now, but then we couldn’t get frustrated that they didn’t work -
L: It was nice to have them.
M: Yeah, and when I first found GIMP I thought it was a fantastic piece of software and it worked and there were big fights about it - libraries, GNOME libraries - I always battled between that and the other one which KDE was using, I can’t remember the name of it now, which is odd because I went for a job there.
L: Did you get it?
M: Yes, I did actually. Actually it wasn’t a job there it was a job at Trolltech - the people who worked at Trolltech had set up a new company - and I was offered a job there, which I accepted. All those fights and arguments drive me nuts, but we have very complex software now, very capable software, very good software and we use it for all sorts of things so maybe now the operating systems do need more complexity and annoyance, or maybe it’s simply that they are trying to make them accessible for everybody... but none of them are. And they shouldn’t be, why should they be? You don’t expect everybody to fix their cars, in fact people are less and less able to fix their cars, like their phones – [picking up his phone] you can’t fix that – you have to take it to experts, I don’t see any problem with experts taking on very technical things.
L: What are your thoughts on distributions like Ubuntu and Mint then?
M: I just never cared. There were lots of different distributions that came out and Debian had some philosophy behind it -
L: They have a social contract.
M: And I’m not interested in that, I just wanted to run the operating system and for it to be as elegant and beautiful as Unix is. When Red Hat became commercial and IBM got involved with them and everyone got involved with them, it was great but they changed it, they added all sorts of crap like systemd. It’s not the first time they’ve done something that was different to what everybody else wanted. There was another big change around sound or something, but it all became a bit fragmented and overly-complicated because underneath all this complication is something that still works perfectly and simply.
systemd
L: I’ve got to mention it, what was it about systemd that annoyed you so much when you first came across it?
M: I’ve never actually used systemd myself so you have to bear in mind that if I did I may think it’s absolutely brilliant and fantastic and like, why didn’t they think of this before? But as I mentioned to you in the car the other day just before hearing about systemd I had been working on a Solaris box, and I think it was the utmp file that was corrupted or one of the user logging files, and Solaris’ dependency boot system refused to start sshd because utmp wasn’t available. It didn’t mount it for some reason, I can’t remember. The dependency software didn’t load sshd so I couldn’t log in to the box purely because utmp was not there and you don’t need utmp to log into the box. And the solution, if you don’t have utmp, is to go mkdir and create a new one! But I couldn’t log into the box. And at the particular organisation you couldn’t just log into a production system, you had to go through all sorts of security gates and have two people with keys to get into a box in order to get the bloody thing to work but I couldn’t log into the ssh. So I didn’t have a very good feeling about startup dependencies to start with.
L: So you thought that trouble was on the way.
M: Well, I just experienced a problem with dependencies that pissed me off. Then I started reading about systemd and I thought, well, what problem are you trying to solve here? And I remember after many many years of seeing things not behaving as they should be and going back through the steps of how that system had booted and going through all the configuration files and I realised I couldn’t do that with systemd. I’m sure you can get the stuff out, the information, but it’s not obvious how to do it but the way the machine boots [with e.g. SysV] is simple and has been around for a long time and it works so well. You can get into dependency problems when something has not started but you tend to know while you’re testing it, not in production because something has stopped behaving correctly. It could happen, I admit that it could, but for me it’s complexity over simplicity and I can’t see that the complexity is giving me anything of benefit.
L: In my experience which is scant in comparison to yours, systemd is less stable. I run four Slackware systems and I’ve never had a failed boot, not once, on any of them.
M: I’ve never had a Linux boot fail.
L: The only time there’s been a failed boot is because I did something wrong.
M: Yes, I have done the same thing myself. I remember my biggest one was at a customer’s, a two day contract, with some government. I can’t remember why but I wanted an enormous sleep running in the background when you boot the machine. There was a bit of a panic, it’s getting late in the evening on the first day and I only had one more day, I booted the machine, it hung, and that was a nightmare. I had no idea why. I remember going home that night really nervous because I had to – had to – get the job finished the next day and I had a machine which had been working and I’d now basically broken it. I remember lying in bed thinking, “oh shit, I bet I left the ampersand off the end of that sleep”. I thought, well if I have, it probably will have booted in the morning... and I got there and it had. That’s probably the most scary moment that I’ve had and I’ve done some stupid things. There was a national computer dealership, I can’t remember their name, and I was working there in their computer room which was small, it was full of IBM computers and me and the other guy, we used to go in there to smoke. I remember fiddling about with a chair and banging my knee against one of the disk drives and it powered off and it shut down the whole country for that dealership.
L: It does make you wonder how many of these unscheduled outages happen at big companies because someone has tripped over an Ethernet cable.
M: That happens a lot, that does happen a lot. I remember hearing a story about someone shutting down an Air Traffic Control system, I think he’d typed ‘shutdown’ instead of ‘man shutdown’ and he got the sack for that. Something like that, something really stupid, something that anyone could do.
Nethack
L: We should wrap up soon because we’re coming up to 75 minutes...
M: Before we do, we do have to mention the most important game, which hasn’t been mentioned yet which has been with me through my earliest career, in fact before I started in computers.
L: Repton!
M: How do you remember that game?
L: I don’t know.
M: No, it’s Nethack.
L: I’ve heard of it, I haven’t played it.
M: Still the most incredible game ever made. A roguelike game, and they’re called roguelike because of the original Rogue. An ASCII adventure game. If you ever get the chance, check out Nethack.
L: What’s so good about it?
M: It is beautiful, it is elegant, it is insanely difficult to complete. I’ve never completed it, I’ve played it for forty years, on and off, I’ve never completed it. It is a Unix game and it’s just beautiful.
L: When you say ‘elegant’ - this is a word I hear coming up – what are you talking about? Elegance of operability, elegance of coding?
M: It’s been a long time since I’ve been involved in all this but I think there was a phrase that people who play Nethack used to use which is “developers of Nethack think of everything”. It’s a very free game, it’s an ASCII character walking around a maze, basically, and there’s lots and lots of little items, magic things, potions, things like that, but you mix them together in a way that nobody’s ever thought of doing that the developers have, and it will react in a way that makes you think “wow, they’ve thought of everything”. It’s an amazing game.
‘Unix philosophy’
L: Looking back, what was your favourite era of computing, what has been the most exciting and interesting for you? Not necessarily what the ‘golden age’ is because that’s subjective – and though this is also subjective it’s more relevant to this discussion.
M: That’s actually quite an interesting question because there have been very specific eras in my computing and they’ve all been very different. I mean, at Baker Street it was great fun turning up at 3am to some accountant, someone who’s got a problem with his computer, in your Land Rover, beaten up old Land Rover, and he’s got a bottle of Chablis in there while you’re trying to fix his computer, it was great. That’s one great thing. Writing system software for national ID systems and fraud detection systems just recently, well fairly recently: it’s fun and interesting but a lot of my answer would depend on who I was at the time as well. It’s all been interesting in different ways and now, right now, when I’m learning CAD software, Fusion 360 and I’m learning Blender for 3D modelling. It’s not operating system level but it’s a good fun thing to be doing. It’s all been different, there’ve been no dull moments.
L: There’s no time which you think it better than another time?
M: No, different, not better. You can have that same feeling as I did when I first got that crack to appear of my space-trading warfare game - multi-user - written in Sculptor report generator, the same feeling of making that work as making a cloth wrap around a solid object in Blender now.
L: That’s kind of an intellectual satisfaction, in the way of learning and accomplishment.
M: It’s problem-solving and seeing the problem fixed. That’s what Linux has always been able to do, and Slackware particularly, because it lays itself open to you and if you’ve got the time and the effort to learn what it’s showing you, you have all the tools you need to do everything and you can find problems and fix problems and solve problems using those tools like you have with Perl or with Python or with C or with Blender or with Dungeon Master. Certainly Nethack - “two dead humunculi” - not “one dead humunculus”, you have one dead humunculus, two dead humunculi!
L: Which systems are you running now at your workplace?
M: Red Hat. We always have some Solaris boxes and one or two Windows boxes but mostly Linux, it’s a Linux shop. We have some SUSE too.
L: So where do you think it’s all going to go? Do you think the user base is going to get increasingly diluted by more people who don’t know and don’t care about their systems? Is it a generational thing? Fewer and fewer people caring about systemd and whether it’s in their computer or not?
M: New people won’t care about systemd because they’ll grow up with it. I think it’s a shame because Unix has been around since the ‘60s in one way or another. It shouldn’t be that something has been around for that long, it has changed of course, but that elegant small tools Unix philosophy, if it’s lost - so many things get lost over time - it would be a shame. Things will still go on. Ultimately I can’t see Linux on the desktop going much further than it already is, but Linux everywhere else and in everything else, yes. My car’s got Linux in it, my phone’s got Linux in it.
L: Your microwave?
M: Can’t be, because it annoys me too much. Everything’s got Linux running in it, or the root of Linux, like Android, its roots are in Linux, iPhone. It’s everywhere, the long battle that’s been around since I’ve been in the industry was between Windows and Unix, and Unix was always looking to be the loser but it turns out it’s very much the winner.
L: When you try to convince a lot of younger users about systemd and why it could be considered a bad thing, one of the epithets that often gets used is the ‘Unix philosophy’. It’s said that a lot of people don’t really understand what it means, so what does it mean to you?
M: It’s just simplicity. Unix is simple and it does what it says it’s going to do and you have total, total control. It never does anything insane or crazy or unexpected. When they say the ‘philosophy’ they mean there’s lots of small tools that do one thing and they do it well and you build them all together to make big tools like you make a house out of bricks, and that’s what people mean by the ‘philosophy’. For me the ‘philosophy’ is possibly more about the elegance of it all, the design of how things are put together. It’s not just systemd that annoys me, I get annoyed by the echo command being an internal bash function rather than an external command - I get why it’s there but for me it loses something for it - and I’ve noticed now that you can tar-*. When you use an asterisk or any kind of similar expression, the shell expands that into the current directory, so if it comes up to a lot of file names they get appended to the end of the command and that’s why they work. But it doesn’t work with 'find', I think it was with find where it doesn’t work, because people don’t remember that and they just think it means all files that match ‘this’ - I’m not explaining it very well because I don’t remember it that well - but now I notice that find has been changed, so that it actually does what you’d expect rather than what it should do, which is break. There are lots of little things like that, things which have been ‘fixed’, tiny little things which have been ‘fixed’ but only because people kept making mistakes whereas what they should have done, in my view, is not make the mistakes.
L: I think what you’re talking about, when you talk about the Unix philosophy, is an ethos. And distributions have ethoses behind them - Debian was all about free software, Ubuntu was about making things accessible and Slackware was about making things as Unix-like as possible - there’s a genuine intention behind that and that’s one of the things that makes it - I don’t want to say a work of passion necessarily because that makes it sound ephemeral - but there’s a genuineness behind it which comes across in how it works and how stable it is. Maybe that’s what’s behind it, that intention behind the philosophy.
M: There are philosophies in different distributions as you said, and that’s why I never got interested in any of them, I was not interested in any of the philosophies of the distributions, I didn’t care, I really didn’t. If I’m honest I didn’t care about the GPL - I’m glad for it, I didn’t care about it at the time - I didn’t care what Richard Stallman was saying that we needed, that we shouldn’t do this, we should do that, we should call it this, we shouldn’t call it that, I admired the guy, like I admired all those people but the only ‘philosophy’ that I personally could identify with was Slackware trying to do things right. That’s the philosophy - for me it never looked like it was trying to stay as close to Unix as possible because Unix is all but disappearing - it was more that it was trying to do it right and remember what was good in Unix, and AU/X started to go a bit funny and move away from what Unix was, but I think Slackware’s always kept it, kept what Unix was always supposed to be. I like that, and if I installed Linux again on one of my machines which, as I said to you, I nearly did the other day, it would be Slackware.
Security
L: With regard to security, people say they’ve never seen a Spectre or Meltdown attack in the wild. People are patching their machines like crazy and using microcode but I hear that there’ve been very few attacks.
M: What I’m going to say is very politically incorrect but I have never suffered from a security attack - well, I may have had an attack but I’ve never suffered a security incident. I’ve seen security incidents and I work in an area currently which is very sensitive to security but I’ve never ever seen anything. We run machines that have uptimes of 1,500 days that we don’t dare reboot anymore. Is that a good thing? No, it’s certainly not compliant with some of the auditory requirements that we’re supposed to be following and those machines are special-case machines, if you manage to penetrate the security in front of those machines - to get to them - in my view you deserve to have access! Because they’re not easy machines to get to, we can’t even get to them within the organisation. So it’s very difficult to say. My personal gut feeling is it’s too much of a religion and I have seen problems introduced by patching, I’ve seen systems break because of patching, I’ve seen new security holes arise because of patching. I’ve been to security conferences where they show you how easy it is to break into machines and no amount of patching is going to fix that and no amount of security scanning software is going to solve the problem.
If you can patch, and you can patch regularly and you can patch reliably and you can patch automatically then you may as well do it, as long as you have a good testing regime in there so you can test the stuff first. I personally, on my phone and on my computers, I make sure I’m not at the latest patch level.
L: You’re not?
M: No, I don’t want to be at the latest patch level.
L: Why not?
M: It’s nearly always broken in one way or another. I don’t like it, I hate it, I hate upgrading stuff on my phone, it’s always worse than it used to be.
L: How much risk is the average home computer user at?
M: No doubt there’s a risk of becoming part of a botnet. Of course there’s a risk, so what Windows do with their occasional patch releases is probably a good thing, but religiously patching with every single patch that comes out? I don’t think it’s worth it. I actually think it’s damaging. If you can afford it and you’ve got the time, it’s not going to do you any harm, well, it’s not going to do you much harm.
L: It’s kind of an “if it ain’t broke, don’t fix it” kind of thing?
M: I’m certainly a big believer in "if it ain’t broke, don’t fix it".
L: So having a 1,500 hour [uptime] -
M: Day.
L: Day, sorry.
M: I’m not proud of it.
L: You should be!
M: Actually in some ways I am proud of it, but the fact is that this machine has been running a national identification system and we have other machines with that kind of uptime and no security breaches ever. There was a security alert on some library, I think it might have been TLS or SSL, some kind of encryption algorithm, I can’t remember, but it was some library, an enormous hole had been found in it and everybody needed to panic about it, the whole organisation and the whole of all the organisations were jumping up and down needing to patch this hole in this library but ours were so old that we weren’t affected by it. It was quite pleasant at the time.
L: People are religiously patching and worrying about the next incremental kernel update or the microcode, they’re obsessed by it.
M: Being on the latest version is never a good idea. Oracle release patches to their database, I think four times a year. Solaris used to release patches four times a year and HP-UX four times a year. There are more people desperately trying to get into these machines now but you never stayed on the latest version so you were always six months behind at best. Because the latest version will introduce bugs. At the place I work now - I think it was SUSE which changed the way that su worked. It took me a while to find out that it was su but we patched it - we had automated patching, if we want to patch we don’t go and patch all our machines individually - and we rolled out the latest versions of patches and some obscure script that somebody had written just broke, and it was an important script, it ran once every six months and it broke. And the reason why it broke was that the guy had written the script properly, really carefully and really well, redirecting standard outputs, standard error, and it didn’t work with su because there was a new feature with su and you had to put some incredibly annoying and fashionably long flag on the command line to get it to behave in the same way that it used to. It’s not a huge thing, I suppose, but we had a production failure because of a patch.
We had a change control system written in Perl at one place, some guy patched another machine which patched Perl and it broke the change management system because it changed the way it handled databases. And the change management system failed. It broke because he had patched the system. It’s not a religious thing to do. I think it’s something to do with care, even with Oracle. The best practice with Oracle is to look through every patch first before you apply it.
L: Slackware -current is a testbed for the next stable. It’s not like the very very latest things get dropped into it, sometimes they get dropped into /testing and then they end up going into -current but -current is for the users to report any bugs or if they need it because it’s got more recent hardware support. But the stable version only ends up having security updates.
M: Yes, don’t keep fixing it until you break it. It’s a difficult thing to say. People do get attacked and machines do get taken over and they do become part of botnets which attack other machines, it does happen. It’s just never happened to me.
L: Have any of your colleagues had serious issues?
M: The only serious security issue I’ve ever known was an enormous security issue caused by the customer doing something really stupid.
L: Can you say anything about what it was?
M: A customer sent us his private key through email. That is a serious security incident.
L: Did anything happen as a result?
M: Yes, we had to create certificates, we had to basically go through a key ceremony again.
L: So some of the most serious computer security issues are PEBKAC?
M: Yes, that is the only time I’ve ever been involved in something that’s been called a security incident or that has been confirmed as a security incident. A firewall, or patching, would not have fixed that customer being stupid.
L: Or PICNIC - Problem In Chair, Not In Computer. OK, we'll probably do more at another point but I think that’s enough for now. Thank you very much.
M: No problem.
3rd Oct 2019. This conversation was fuelled by a respectable quantity of beer, wine, e-cigarettes and dog noises.
Total Comments 0
