The Shadow Of Tradition

In the ancient Italian village, where the roots of tradition grew deep and unyielding, there lived Giovanni, a venerable chef whose soul was intertwined with the art of timeless Italian cuisine. His dishes were more than mere food; they were hymns to history, odes to the old ways.

One gloomy evening, as a foreboding wind moaned through the cobbled streets, Giovanni overheard a disturbing conversation. A foreign tourist, her demeanor unsettlingly fervent, spoke of a culinary aberration so bizarre it seemed almost heretical – using Nutella, the sweet hazelnut spread, as a pasta sauce.

The notion was not just absurd to Giovanni; it was sacrilegious. But there was something more, something unnerving about the tourist’s zeal. Her eyes held a dark glint, her words tinged with an ominous reverence.

Compelled by a sense of foreboding curiosity, Giovanni found himself at the tourist’s gathering that night. The villa, usually a haven of warmth, felt different – shadows seemed to stretch and twist in unnatural ways, and the air was thick with an unspoken tension.

As the “Nutella pasta” was served, the atmosphere turned stifling. The dish, with its dark, swirling sauce, seemed almost alive, a black maelstrom on a plate. Giovanni, his heart pounding, tasted the forbidden fusion.

The flavor was disconcerting, a chaotic clash of sweet and savory that seemed to mock the very foundations of Italian culinary principles. But it was not just the taste that disturbed him; it was the reactions of those around him.

The guests, under the tourist’s watchful gaze, ate with a fervent intensity. Their expressions were twisted into grotesque parodies of enjoyment, their eyes vacant yet burning with a strange fervor. The tourist herself, her smile wide and unnatural, whispered cryptic praises to an entity she called “Cthulhu.”

The word sent chills down Giovanni’s spine. He remembered old tales, whispers of ancient, malevolent beings lurking in the depths of the unknown. The tourist’s devotion, her eerie chant of “Cthulhu fhtagn,” resonated with a sinister energy that filled the room.

As the night wore on, the villa seemed to pulse with an unseen force. The shadows grew darker, the laughter more hollow. Giovanni felt as though he was teetering on the edge of an abyss, staring into a culinary and existential void.

He left the villa feeling a deep, unsettling dread. The village, once a bastion of tradition, now felt like a stage for a cosmic play, a place where the boundaries between the mundane and the unfathomable blurred.

In his kitchen, Giovanni pondered the surreal experience. The Nutella pasta was more than a dish; it was a ritual, a dark offering to a god from beyond the stars. The tourist, a cultist of this ancient entity, had introduced not just a recipe, but a symbol of a chaotic, otherworldly influence.

Haunted by the night’s events, Giovanni knew that the culinary world he cherished was under threat, not just from changing tastes, but from forces beyond human comprehension. The shadow of Cthulhu loomed over his beloved traditions, a reminder that some doors, once opened, can lead to realms of terror and madness.

The Matrix

With Matrix IV just around the corner, I re-watched the original trilogy. The first movie has held up very well, it’s just as good as I remembered, a perfect slice of 90s nostalgia or zeitgeist or whatever. But I had forgotten just how excruciatingly bad the second and third were. Everything about them is bad, and not in a good way. The dialogue and acting are risible. I actually felt sorry for the actors having to deliver them. The action sequences are tedious and boring, I skipped through most of them. New characters are introduced cold and we’re supposed to care about their fates, which existing characters somehow become less developed, which shouldn’t be possible. Even by my extremely loose sci-fi and fantasy standards, the plot makes no sense and has no continuity from scene to scene, things just happen, and then it’s onto the next unrelated thing. Really, those two films together ought to have ended the careers of everyone involved, except of course for Keanu, Fishburne and Belluci. It boggles the mind how the producers ever got funding for a fourth.

After WNC/COVID-19

Our dear leader Boris has stated that if everyone follows the guidelines, we might stand a chance of dealing with the local outbreak of Wuhan Novel Coronavirus in 12 weeks. I think it’s pretty safe to say that that is the minimum duration of this crisis as in, the guidelines will remain in place at least that long. But alot can happen in 3 months and it’s not too soon to wonder what sort of a world we will emerge into afterwards. Because this has the potential to alter everyday life on a permanent basis.

One example is that everyone who can work from home will be fully adapted to doing so in that amount of time. I have done it on and off for years so I was already very well set up for it in terms of ergonomics and logistics (daily routine), a few people I know are scrambling now to acquire docking stations, large monitors, webcams for videoconferencing, office chairs etc but they will have those in a week at most, as Amazon seems to be working perfectly. A Nespresso machine too perhaps! It might take a little longer to build up to the daily routine, get used to working during business hours and avoiding any distractions around the house but not too much longer. In mid-late June people may be desperate to go back to commuting on the god-awful British trains… Or they may decide to WFH permanently. If it’s working for companies, if the workforce has adapted and productivity only took a temporary hit, will those companies want to go on paying for office space? The knock-on effects of that will be huge and felt at every level of society, all the way from hourly workers who serviced those offices, to massive investment portfolios who will see the value of commercial property tumble.

Will there be pubs and restaurants to go to? Those that relied on office workers for lunchtime or after work are probably done for. They can’t shift to food delivery or takeaway because they are nowhere near their former customers. Local establishments probably can but again that will be very different. Alcohol delivery is already sewn up by the supermarkets (those that can manage to stay online that is). But assuming it does blow over then it’s an idyllic lifestyle of people working from home popping into their local for a spot of company during the day, or coming in the evening for a good few drinks, not “just the one” because there’s a train to catch, and it’s just a short walk home and no commute the next day either, so no need to wake up early. If they survive the initial slump local pubs and restaurants could do very, very well in the new world, fingers crossed for them. Hotels that catered to a business clientele are going to have to adapt if they can, and will have to deal with the other knock on effect of without business travel and its accompanying economies of scale, air travel will become less accessible. People might miss that but they won’t miss commuter trains!

Speaking of trains, public transport is a huge disease vector. Will we see people going back to cars as the preferred means of transport? It will be a hard sell for the government to encourage the use of buses, having just fought a pandemic that in all likelihood was spread by them – even after the guidelines came out bus passenger numbers in London barely moved, whereas Tube numbers sharply declined.

Another positive that could come out of this – and I am less convinced this one will happen, but I hope – is better working condition and remuneration for those who by the nature of their work, cannot work from home. It is very interesting, we had a supermarket delivery last night that we had already booked and it arrived on time with only a few items missing, despite that supermarket’s website having collapsed under the load a week ago, and still down. So the people who cannot WFH, the warehouse workers and delivery drivers are doing their jobs, whereas the ones who can WFH, the website programmers could do better.

If you’re an engineer, skills in VDI, VPN, and all the related things are likely to be in high demand going forwards, and skills in traditional office equipment and PC maintenance are not. If you’re a user, get used to being more self-sufficient! You will soon get used to it!

RIP Gobbolino

This slideshow requires JavaScript.

Our beautiful and much-loved cat Gobbolino, 23^rd July 2000 – 28^th December 2019. An intrepid exploring cat in her youth, as she got older, she preferred her familiar places with all of her important things nearby. So we brought the world to her. She loved to investigate all my kit when I returned from wild places with interesting smells, she was curious about anything you brought to show her, she liked to play with my iPad including taking a selfie once. She was such an inquisitive and talkative and affectionate cat that we noticed straight away that evening when something had suddenly gone badly wrong. We rushed her to the emergency 24-hr vet, at the cattery she used to stay at when we were away, B driving and Gobbolino sat on my lap. I told her they would make it all better and she would be back home again soon, and I genuinely believed it as I said it, but it was already too late. All the vet could do was to take the pain away.

I get that some animals – and some people – are ready to die, that their quality of life has deteriorated so far that death would be a welcome relief. Technology can extend physical life far beyond what is “natural” regardless of the mental cost. But that was not our Gobbolino, even at 19½ she was still actively engaged with and interested in everything around her, she enjoyed her food and selected company, she wasn’t in any chronic, debilitating pain. She wanted so desperately to live but her internal organs were just worn out beyond repair, by the time we reached the vet half of her body had already shut down. Praise be to Aslan that at least it didn’t happen while we were away or at work, I can’t bear the thought of her dying slowly in agony with no one there.

I don’t want to lose any memories, especially those of her very final moments, but I don’t want that frantic last hour to be my overriding memory of her either. I want to remember her beside me on the bed or the couch or behind the monitor in the study, purring like a diesel engine. I want to remember the funny bouncy way she walked on stairs. I want to remember the way she would look me in the eye and emit a complex sequence of meows, squeaks and chirrups, obviously expecting me to understand every word. I wish I had some recordings. I want to remember the way every year she would just become incredibly fluffy with her winter fur. I want to remember the way she and her sister Sootica looked after me when I had broken my leg and ankle. I want to remember that too much fuss and attention made her self-conscious. I want to remember that I could never resist her pleady eyes when she wanted something, and she knew it, much to B’s amusement. I want to remember her love of fried eggs, she had one for breakfast on her last morning, from her own labelled box, and a tiny piece of bacon. And a hundred more things…

Her and Sootica were always searching for the perfect sunbeams to laze in (the quest continues for Sootica). When they found one, an hour later the sun would have moved and then they would come to me and complain about it. I’m sorry I couldn’t move the sun for you, no matter how much you believed I could, and I’m sorry I couldn’t hold back time either.

Password Strength Is A Hard Problem

We are all familiar with the classic advice on password strength, a mix of lowercase, uppercase, symbols, numbers, of a certain length. It is trivial to construct a validator for password strength in Python

>>> from string import ascii_uppercase, ascii_lowercase, digits, punctuation
>>> def check_pw1(p, m=8):
...     return len(p) >= m and all(map(lambda x: any(y for y in x if y in p), [ascii_uppercase, ascii_lowercase, digits, punctuation]))
>>>

The problem with this is that you can meet all these requirements and still have a bad password, for example:

>>> check_pw1('Gaiu$123')
True
>>> check_pw1('P@55w0rd')
True
>>>

When substituting symbols or numbers for letters, everyone does it in predictable ways, such as S → $ or 5, T → +, O → 0 and so on. So these offer only minimal protection against brute-force dictionary attacks, or even intelligent guesses. We can easily generate a few variations (there are obviously combinations of variations too that would need to be considered too†):

>>> s = {'s': ['5', '$'], 'a': ['@',], 'o': ['0', '.']}
>>> def subs(a):
...     x, y = a
...     return [(x, z) for z in y] + [(x, x.upper())]
>>> l = ['password'.replace(a, b) for a, b in [x for y in map(subs, s.items()) for x in y]]
>>> l
['pa55word', 'pa$$word', 'paSSword', 'p@ssword', 'pAssword', 'passw0rd', 'passw.rd', 'passwOrd']
>>> p = 'password'
>>> for a, b in [x for y in map(subs, s.items()) for x in y]:
...     p = p.replace(a, b)
...
>>> p
'p@55w0rd'
>>> l.append(p)

Looking at the size of the dictionary, with a few combinations you would end up with well over a million possibilities, possibly several million, but remember that with techniques like rainbow tables these kinds of attacks were feasible decades ago, now you can easy throw 10000× the compute power at it.

gaius@klossy:~$ wc -l /usr/share/dict/words
102401 /usr/share/dict/words

Maybe we can do something a bit cleverer using NLP techniques:

>>> from nltk.metrics.distance import edit_distance
>>> def check_pw2(u, p, m=8):
...     return len(p) >= m and edit_distance(u, p) >= m
...
>>> check_pw2('gaius', 'Gaiu$123')
False
>>> check_pw2('gaius', 'P@55w0rd')
True

But it’s still not foolproof.

>>> def check_pw3(u, p, m=8):
...     return len(p) >= m and edit_distance(u, p) >= m and p not in l
...
>>> check_pw3('gaius', 'P@55w0rd')
True
>>> edit_distance('password', 'P@55w0rd')
5
>>>

Hah, I’m not as clever as I thought I was, in building my list of substitutions I “forgot” that every letter can also be substituted for its opposite-case counterpart. And you can bet any users of my system, having to change their password regularly enforced by policy, would quickly discover that and use it as a means to minimise variation each time to make it easier to remember. I could then check the edit distance between the password and every word in the dictionary… It’s a neverending game of cat and mouse, and only serves to annoy everyone.

>>> from timeit import timeit
>>> timeit("""from nltk.metrics.distance import edit_distance
... f = open('/usr/share/dict/words', 'r')
... for w in f:
...     edit_distance(w, 'P@55w0rd')
... """, number=10)
244.13386648699998

But we can try it:

>>> check_pw4('gaius', 'P@55w0rd')
abactor

False
>>> check_pw4('gaius', 'agi395wk%')
aardvark

False

Finally! But maybe it’s too good!

>>> check_pw4('gaius', 'agi395wk%', 7)
agile

False
>>> check_pw4('gaius', 'agi395wk%', 6)
True

OK, we can maybe get decent results relaxing it a bit. It still needs tweaking but I have proved my point! Programatically checking a password short enough to be memorable is a hard problem! It would require an almost impossible effort to cross check against names of relatives, names of pets, birthdays, postcodes, car registrations… A password can simultaneously be both secure in theory and trivial to guess in practice.

You have to get into the realms of pwgen 20 to be secure, eventually you end up with something as long as an SSH key! People are too predictable and the traditional rules are too easy to workaround. I wonder if any formal studies have ever been done into the most complex password a normal user can cope with, or whether “the rules” were just dreamt up by someone one day without really thinking about it. You very quickly reach the tipping point which leads to post-it notes on monitors!

None of this matters anyway really, I’ll wager very few cyberattackers make the initial breach via guessing a password these days. Just waltz in through an unpatched vulnerability and grab the whole user database and crack, sorry, “recover” it offline at your leisure… 😀

† Not my intention here to give away anything actually useful obviously

Microsoft Professional Program Cybersecurity

In a previous job, amongst many other things I was responsible for the tending of a small herd of HP DL-whatever servers, ~1500 machines all running RHEL. These were considered mission-critical for the business unit. I used various tools for this, one of which was the Integrated Remote Console. This would be used when some hardware failed, which happened surprisingly often; HP quality is not what it was in the good old days of PA-RISC and HP-UX. But I digress.

One day I came into work and found that I had been locked out of various systems and that my name was on a spreadsheet sent to “senior management” because I had allegedly been caught “participating in hacker chatrooms”. I did try to explain to the cybersecurity team that actually this IRC.EXE was nothing to do with that but they were soooo pleased with themselves for having busted a hacking ring, or so they thought, that they wouldn’t listen. So I went and spoke to the head of the business instead, and told him that I could no longer support his mission-critical platform. Well that got sorted out very quickly. I never got an apology from the cyber guys, but they never bothered me again. Incidentally, the ad for this job had even included the phrase we’re looking for real hackers, using the more traditional meaning of the word :-)

I could tell a dozen stories like that from various points in my career. Cybersecurity people are a very mixed bag. Some are very good indeed and can reason about complex systems and their failure modes in ways that had never even occurred to me before. But some, lacking the background of software or systems engineers, have no context and no real idea about what the systems they are supposedly defending actually do, and what’s normal, legitimate activity and what’s suspicious, so they blunder around like bulls in a china shop. And this latter group, for some unknown reason, see themselves as an elite, aloof from common engineers, which doesn’t help anyone – security is a team sport, and for it to work, everyone must be a player, including engineers and end-users. Understanding how people use the systems day-to-day is very important, unless security seems natural to them, people will try to find workarounds for it, like USB drives, Dropbox, leaving their unencrypted laptop in the car…

Anyway I do have such a background, and guys like me have always done the bread-and-butter of cyber (or infosec as it used to be called) and now I have completed a course to round out my knowledge of this field too:

What do I mean by bread-and-butter? Well, most security work isn’t especially glamorous and exciting. Keeping the hardware and software inventory up to date, staying on top of newly discovered vulnerabilities and newly released patches from vendors, triaging them into change management, ticking off checklists, administering routine updates of AD or similar, maintaining systems that collect and scan logfiles, dealing with false positives (manual investigation) and fine-tuning the triggers, probing our own systems with fuzzers and suchlike, archiving things for compliance… Educating and if necessary enforcing good security practice throughout the organisation. Identifying requirements, evaluating solutions, presenting the findings, same as any other product or service the organisation might use. The occasional forensic analysis if there is some possible indication-of-compromise. Reporting on all of this to “key stakeholders”.

It’s important work and it needs to be done but it’s also, for most of us, on top of our real jobs. The full-time cyber guys are off doing… whatever it is they do all day. I’m kidding. Sort of. I think most outsiders think that that is what the entire field is!

---

Unfortunately, Microsoft have just announced that they’re retiring the MPP so this will be my last one (I was going to do the IoT track next year). That’s a real pity because MPP was one of the few that taught conceptual and theoretical skills along with hands-on technical, and would thus retain long-term value even when the specific tools used on the course were superseded. That is what attracted me to it in the first place. The new “role based certifications” are purely about operating particular versions of particular products, which is a big step backwards to short-term value only. It was a bit sudden as well, there will be many people who, with other commitments and so on, will struggle to finish by 31^st Dec. I spent nearly a year doing the Data Science track, on and off. I did this one quickly because, as I mentioned, alot of it was already familiar!

For posterity’s sake I will preserve the MPP Cybersecurity curriculum here:

1. INF246x: Enterprise Security Fundamentals
2. INF249x: Threat Detection: Planning for a Secure Enterprise
3. INF250x: Planning a Security Incident Response
4. INF251x: Powershell Security Best Practices
5. INF258x: Windows 10 Security Features
6. INF259x: Windows Server 2016 Security Features
7. DAT243x: Securing Data in Azure and SQL Server
8. INF253x: Managing Identity
9. INF260x: Microsoft Azure Security Services
10. INF261x: Microsoft Professional Capstone : Cybersecurity

A good mix of general principles and technical specifics, even the particular Windows courses covered material that was applicable to other platforms. I even learned some new things about SQL Server! The capstone involved defending a mixed network of Windows 8.1, Ubuntu 14 and Windows Server 2012 from a simulated attack in real time – with the older platforms alot of the more modern tooling from Windows 10 and 2016 was not available so you had to use your wits. Fun!

I suppose I’ll have to do the same for the other ones as well. I really hope Microsoft will reconsider.

Going forwards, this was an entirely defensive course, and since I now have to look beyond Microsoft for CPD, maybe I’ll try something offensive.

Forgotten Features

The ancient wizards who defined the ASCII standard knew what they were doing. ASCII for those who have not come across it, is the standard means of encoding mainly textual data as a stream of 7- or 8-bit bytes¹ for transmission or storage. It’s how your computer works inside. So the character A has value 65 which is 01000001 in binary, on the wire or on the disk. Codes 32-126 are printable characters like these you are reading. Characters 1-31 are special, they are called control codes and one use for them is for devices to communicate instructions, such as a device reading code 10 knows to advance to a new line, or code 8 to move one space backwards (these are how your ↵ and ← keys work, under the covers). Some of them don’t really make sense in the modern context, carriage return (code 13) in its original usage would cause the receiver to physically move a dot-matrix printer head back to the left and line feed would physically advance the fanfold paper to the next line, but nevertheless the codes are still there and modern operating systems know how to interpret them for modern devices.

ASCII control codes also include values for structuring ASCII files or streams of bytes, for example to represent tabular data. The important thing about them is that they are outside the range of values that they delineate, so their meaning is always unambiguous. Code 28 is file separator, 29 group separator, 30 record separator, 31 unit separator. So it is easy to encode one or more tables of data in one chunk of ASCII. Or at least it should be but…

There is no “CSV standard”, so the format is operationally defined by the many applications which read and write it. The lack of a standard means that subtle differences often exist in the data produced and consumed by different applications. These differences can make it annoying to process CSV files from multiple sources

Everyone seems to have entirely forgotten that these exist! There is nothing weird or exotic about them, the first edition of the standard was published in 1963 and ASCII has been baked into nearly every computer ever since! Everytime I need to deal with CSV files², a format which is full of edge cases that noone can agree on, I despair a little at the state of the profession and our claims of being software engineers. And that’s before we even get onto more recent wheel-reinventing like XML, JSON, YAML… Anything that uses normal printable characters³ to delineate records or otherwise impose structure becomes unwieldy as soon as you want to have one of those characters in the data, for a start. Everyone who has had to deal with angle brackets or ampersands in HTML has been bitten by this at one point! I suppose one point in CSV’s favour is that at least each delineator is only one byte, unlike the others which have a great deal of overhead.

¹ The extra bit could be used to get another 127 chars, or for error detection.
² Many times per day
³ There’s no reason a text editor couldn’t display something for the control codes, when it can easily show paragraph breaks as ¶ or whitespace as ◊.

Learning a New Language

Generally, every program I write, regardless of what useful thing it actually does, and regardless of what programming language it is written in, has to do certain things, which usually includes

• Importing a library and calling functions contained within that library
• Handling datatypes such as converting between strings and integers, and knowing when this is implicit or explicit, how dates and times work, and so on
• Getting command line parameters or parsing a configuration file
• Writing log messages such as to files or the system log
• Handling errors and exceptions
• Connecting to services such as a database, a REST API, a message bus etc
• Reading and writing files to the disk, or to blob storage or whatever it’s called this week
• Spawning threads and processes and communicating between them
• Building a package whether that’s a self-contained binary, an RPM, an OCI container, whatever is native to that language and the platform

It’s easy to find examples of most of these things using resources such as Rosetta Code and my first real program will be a horrific cut and paste mess – but it will get me started and I’ll soon refine it and absorb the idiomatic patterns of the language and soon be writing fluently in it, and knowing my way around the ecosystem, what libraries are available, which are the strengths and weaknesses of the language, the libraries, the community and so on. Once you have done this a few times it becomes easy and you can stop worrying so much about being a “language X programmer” and concentrate on the important stuff, which is the problem domain you are working in.

ML in the Real World

About a decade ago now, I was doing a lot of what we would now call ML†, using the what is now called data exhaust‡ from the production infrastructure of an exchange, both the OLTP and DW sides. It was simple timeseries stuff, just lots of it. I could look at the storage arrays, say, and make very accurate predictions about when some threshold would be breached, very far in advance. I could get from the ticketing system when a purchase order for more capacity was raised, and when it was fitted, and say exactly when to place the order with the vendor to get the parts delivered on time. Same with the time taken to fetch a tape from offsite. I looked at batch job completion time vs CPUs, not only did I know well in advance when we would need more, my algos had worked out for themselves that there were periodic spikes such as end-of-month reporting, and knew that there was no need to alert. All sorts of stuff like this, I thought it was pretty clever and I was quite pleased with myself.

In practice tho’, no-one cared. We went on ordering more disks and shelves when the dumb Nagios alert fired, so long as it could be added before there was an actual, during-trading-hours outage, that was good enough so why change? We added more CPUs when the moaning of the analysts reached the ears of the CEO and he in turn moaned to our boss about it, there was no formal SLA on job completions. And everyone who had been there 6 months or more knew which alerts to ignore and simply did that, no-one even bothered to blackout them (also, because there was nothing that could be done about them anyway).

I had a lot of fun doing all this, and I learnt a lot, this was the time I got seriously into Python, NumPy, Matplotlib and so on, skills that have served me well ever since, and applied linear regression, PCA, and various other techniques, to real data. But the real lesson is, if you’re going to try to use ML in the real world, you have to use it to solve a problem that you actually have, and generally, existing problems already have a solution that is good enough that ML doesn’t tell anyone anything they hadn’t already figured out themselves or was already embedded in institutional knowledge. Maybe if we didn’t already have industry-leading uptime and transaction volumes on human intuition alone, it might have been taken more seriously. I think many if not most ML practitioners are going to run into this scenario at some point, and need to have a story ready, which I didn’t.

† It was just called applied or predictive statistics back then
‡ It was just called metrics back then, or logging, gotta keep up with the buzzwords!

Article 13

A lot of fuss is being made about the potential impact of so-called Article 13 on YouTube. I think there are two possible outcomes that would be acceptable to me. Either:

• YouTube verifies the identities of all uploaders, and the uploader is fully responsible for the legality all new content, relegating YouTube to the status of a common carrier. After a brief grace period, any pre-existing content that isn’t claimed by a validated user is purged.
• YouTube moderates† every upload before it is publicly visible and is responsible itself if it re-publishes anything that is illegal, and of course for any and all pre-existing content that they continue to publish without having retro-moderated.

Their current position, which seems to be that they can’t make as much profit as they’d like with the overhead of complying with the law, isn’t really justifiable IMHO.

† This could be done with AI, if they are willing to pay the fine/compensation to the real copyright holder, when it gets it wrong.