What is Experience?

Like many people of my generation, I started programming at an early age. At 10, I was writing 6502 assembly language. My beloved BBC Micro lacked built-in flood fill, circle and ellipse drawing, and other graphics commands, and I spent many happy hours working on these. I used interrupts and vectors to place a clock in the corner of the screen, of limited use since the Beeb lacked a battery-backed RTC, but, multitasking baby!! Exciting stuff. I dabbled in making games, I did a little 3D and simple physics modelling, I wrote my own version of LOGO†.

I sometimes come across people with similar backgrounds, who count their programming experience as starting from those days‡. Now it is natural that I would feel an affinity here, but real experience has taught me that that experience doesn’t carry over into programming in a commercial setting. The problem is, a normal, working programmer, does an awful lot of things other than write code. Hobby projects don’t teach you to write documentation, or specs for new features, or clear bug reports that anyone can pick up to reproduce the issue. It doesn’t teach you how to work in a team, how to branch and merge, how to do code reviews, how to mentor others. It doesn’t teach you how to manage priorities or dependencies or balance conflicting demands. It doesn’t teach you how to talk to users or sponsors or vendors. It doesn’t teach you how to make intelligent use of technical debt. It doesn’t teach you about how to dive into unfamiliar code, under a tight time constraint, in a business domain you don’t fully understand, and make your change without breaking anything else (or what to do if you do). Dealing with distributed, heterogeneous systems can make things get very complicated, very quickly, in ways you can’t anticipate unless you’ve done it a few times before. There are a million other things… A guy or girl who does this isn’t even exaggerating their experience, they’re demonstrating that they don’t even understand what experience is. People in their early 20s who think their “10 years” experience puts them on a level with someone in their 30s who has 10 years commercial experience plus their teenage and other personal hobbyist projects. Please stop this!

† Then about 14 I discovered other interests and didn’t do much computing again ’til college at 18. FORTRAN baby!!
‡ Or equivalent for their generation.

Posted in BBC, Business, Random thoughts | Leave a comment

Form vs Function in Applications

When you have been around the industry for a little while, you will see that nearly all applications, especially user-facing ones, are made of only three kinds of things: Forms, Reports and Workflows. A Form is simply an interface, a screen or a page, for the user to input some data. It is made of text fields, checkboxes, drop-down menus, and so on. A good Form offers prompting and validation, such as a text field that is supposed to be a number is actually a number and within a sensible range. It can be dynamic, updating fields depending on each other, all sorts of fancy things, and when the user is done, saves the results in a database.

A Report is a screen or page that accesses a database, often one populated at least in part by Form input, filters, sorts and aggregates the records, and presents them in a useful way, for example a graph or a table. Often Reports have a few Form-like elements too, so the user can refine or drill down into the data. A Workflow simply links Forms and Reports together. Behind both you might have Services (APIs). So 4 things, to make (nearly!) any application you can think of.

Let’s consider a website like Amazon. You go to it, and it runs the default Report. You refine that Report into a selection of things you’re interested in, and select one, fill in a Form to buy it, and the database is updated, one item transferred atomically from their inventory to your basket, by calling a Service. Then in the warehouse someone runs a Report that tells them what to pack and who to ship to, then they fill in the Form to update the status. There’s nothing in this, fundamentally, that couldn’t have been done on a 1971 IBM 3270, it just might not look as pretty to modern eyes, but all the functionality and interactivity would be there. If you’re buying a book, do you really need a photo of the picture on the cover to decide?

Once you start thinking at this level, suddenly it doesn’t matter that the lifetime of the current fashionable web page generation language is only a few years, or the latest Javascript framework will only be around for a few months. Let junior programmers (who I’ll define as those who define themselves primarily in terms of their language rather than their domain) worry about that and worry about running just to keep up with their peers jumping on the latest bandwagon. Concentrate on the meat of the application, and use tried and true languages and platforms to do it. This isn’t an anti-technology rant by the way, just a piece of advice for those ready for the next level.

Posted in Business, C++, Oracle | Leave a comment

Can anyone learn to code?

Anyone can learn to code, we are constantly told these days, but is it true? Yes… in the same sense that it is true that anyone can learn to play a musical instrument. Many people do, and many of them get a great deal of personal satisfaction from it, play in bands, write songs, meet new people, and lots of other good things.

But if a man selling guitars and guitar lessons tells you that playing the guitar is a guaranteed route to a well paid, secure career, then you would be wise to take that with a pinch† of salt. Or worse, if he tries to convince you that your kids should just learn to play the guitar and join a Silicon Roundabout startupband, instead of studying traditional subjects.

Similarly anyone can learn to cook, but the skills required to cater a dinner party for some friends don’t scale up to catering a formal reception for hundreds, or running a restaurant. Not even Jamie Oliver is stupid enough to try to convince anyone his books and TV shows will teach you that. If schools taught people to cook meals from basic ingredients then as a nation we would be healthier, wealthier and happier, but they can’t even manage that… Why does anyone think they can teach “coding”?

† Handful

Posted in Business, Random thoughts | Leave a comment

2014

So, how was 2014 for me, in tech at least? Pretty good. It has been quite an interesting experience adapting to an all-proprietary tech stack (database, language, IDE, job scheduler) but I feel I am finally getting a grip on how all the moving parts hang together and how the sum is greater than the whole of the parts. So, professionally, I’m pretty happy and I find out early next year how happy they are with me :-)

The systemd farce has prompted me, after nearly 20 years, to look at alternatives to Debian for my personal Unix(-like). Candidates for a replacement so far are DragonFly, Minix3 and Debian/HURD, tho’ I have to wonder how viable the latter is. The Debian project simply doesn’t have the manpower to maintain two parallel distros, once systemd has fully infected Linux. Windows 95 called, it wants its registry back. I have these running in VMs to kick the tyres right now. But more and more of my scarce time for personal projects is taken up with BBC Micro and Atari ST stuff now anyway…

Speaking of the Beeb, Elite: Dangerous is available now. I have to admit I was skeptical that it would ever ship, and I’ll hold my hand up to say I was wrong about that, good work Braben. However I won’t be playing. The lack of a single player mode is the deal breaker for me – you can play single-player but you are forced to participate in the shared universe, which runs in real time. But in a game I want the universe to stop when I’m not there and resume where I left off whether that’s an hour later or a month. And consider this: I bought my copy of the original Elite, in 1984, with pocket money I’d saved. Acornsoft ceased to exist as a separate entity in 1986 as the bottom dropped out of the home computer market. But I still play it in 2014, on original hardware. Should something happen to the servers owned and operated by Frontier, what happens to online-only Elite?

Nothing to report on the OCaml front. I’m still into it, but just haven’t had time to do much on OCI*ML. As I said previously, I had gotten it to the point at which it was useful for the work I was doing at the time. Features such as handling BLOBs are still outstanding, maybe I will have a chance to add that (and find a way to do it without my company claiming the IP). I’d like to maintain my connection to the OCaml community, because you never know… And I need to get properly up to speed on C++14, again to keep options open. (RIP Dr Dobbs). Will I ever return to the world of Oracle? I won’t rule it out but what I am doing now is a radically different way of thinking about databases and the applications that run on them; a few years of this can only strengthen my database and development career long term.

My prediction in wider tech in 2015 is this is the make-or-break year, either something is done about impossible-to-secure HTTP, SMTP and TCP/IP networks, or the public massively disengages from online services in general, sticking only to those websites with a) a good track record of security and b) an even better track record of making things right financially and logistically when their customers are impacted by a breach.

Posted in BBC, C++, Linux, Ocaml, OCIML, Oracle, Random thoughts | Leave a comment

2013

Long time since I have updated here, a lot has been happening. For a start part 4 of my series on Oracle 12c new features is unlikely to be written since in October I started a new job which is non-Oracle. I’m now working for an organization with specialized enough needs to have written its own in-house database, the raw building blocks of which are Linux, C++ and Python, integrating soft-realtime data access, replication with flexible topology, sophisticated batch scheduling and a whole range of other features, globally distributed and operating 24/7. My new role is Application Development Lead.

It is a bit strange; for the last 15-odd years Oracle has been my bread-and-butter, from versions 7 → 11g in Prod and 12c in Dev, as a DBA and a developer, but it is good once in a while to step outside one’s comfort zone. Here’s wishing for an exciting and prosperous 2014 for all!

Posted in Oracle, Random thoughts | Leave a comment

Real World OCaml

In the course of my work with OCaml I have traditionally resisted using anything other than “pure” OCaml, and the facilities of the underlying OS. So rather than OMake or OASIS I just used plain, old-fashioned Makefiles. For package management, I relied on APT on Debian and MacPorts on OSX. And I avoided both Batteries and Core. No so much out of a fear of “backing the wrong horse” but just to make whatever I did as portable and easy to adopt as possible. And also, in the early days, I didn’t really know enough to choose anyway, and I wanted to work with the raw language rather than a high-level framework. Sort of like you can learn to program MFC without ever really learning C++.

But now Real World OCaml (which I have on pre-order) is in final draft, and spent some of yesterday getting my Debian and OSX environments set up for it†. One quirk I quickly found is that both have pkgconfig as a prereq, which for whatever reason, neither system had already, and that’s not mentioned on the page, maybe everyone else has it by default. I have a bunch of OCaml stuff in-flight at the moment – OCI*ML test suite and new features, some playing with Project Euler (solved 1516 problems at time of writing) and now working my way through this (trying not to skip to FFI which is a keen interest of mine, obviously!). That’s on top of playing with Oracle 12c, and I have barely started properly playing with C++11 new features yet!

† Links to the draft of the book will stop working at some point I expect…

Posted in Ocaml | 7 Comments

Strange Datetime Problem

While working on my unit tests, I came across a sporadic failure in inserting and selecting Datetimes to the database, so I wrote a quick test harness to see what’s going on:

open Ociml
open Printf

let () = 
  let lda =  oralogon "ociml_test/ociml_test" in
  let sth = oraopen lda in
  for i = 0 to 100 do
    orasql sth "truncate table test_date";
    let d = (Datetime (localtime (Random.float (time() *. 2.)))) in
    oraparse sth "insert into test_date values (:1)";
    orabind sth (Pos 1) d;
    oraexec sth;
    oracommit lda;
    orasql sth "select * from test_date";
    let rs = orafetch sth in
    match (rs.(0) = d) with
    |true -> print_endline (sprintf "Inserted %s, got %s, OK" (orastring d) (orastring rs.(0)))
    |false -> print_endline (sprintf "Inserted %s, got %s <-------- FAIL" (orastring d) (orastring rs.(0)))
  done

This fails about 3% of the time, for reasons I cannot fathom, there seems to be no correlation with summer time. Here’s a set of results incase anyone else can figure it out:


gaius@debian7:~/Projects/ociml$ o
        Objective Caml version 3.12.1

	OCI*ML 0.3 built against OCI 11.2

not connected > #use "tests/date_grinder.ml";;
Inserted 31-Jul-1993 20:21:44, got 31-Jul-1993 20:21:44, OK
Inserted 01-Mar-2022 09:17:53, got 01-Mar-2022 09:17:53, OK
Inserted 18-Jan-1995 05:48:57, got 18-Jan-1995 05:48:57, OK
Inserted 24-Jul-2024 14:10:44, got 24-Jul-2024 14:10:44, OK
Inserted 12-Jan-1991 12:32:01, got 12-Jan-1991 12:32:01, OK
Inserted 03-Nov-2018 18:26:46, got 03-Nov-2018 18:26:46, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 08-Jan-2036 17:58:34, got 08-Jan-2036 17:58:34, OK
Inserted 31-May-2001 07:29:34, got 31-May-2001 07:29:34, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 07-Aug-1986 04:59:54, got 07-Aug-1986 04:59:54, OK
Inserted 08-Mar-2036 15:19:15, got 08-Mar-2036 15:19:15, OK
Inserted 23-Mar-1975 09:36:54, got 23-Mar-1975 09:36:54, OK
Inserted 26-Aug-1998 10:39:15, got 26-Aug-1998 10:39:15, OK
Inserted 23-Jan-1985 13:23:09, got 23-Jan-1985 13:23:09, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 02-Dec-2024 12:06:13, got 02-Dec-2024 12:06:13, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 20-Mar-1985 11:47:15, got 20-Mar-1985 11:47:15, OK
Inserted 25-Oct-1976 20:16:19, got 25-Oct-1976 20:16:19, OK
Inserted 29-Dec-1972 23:46:42, got 29-Dec-1972 23:46:42, OK
Inserted 17-Aug-1993 06:41:06, got 17-Aug-1993 06:41:06, OK
Inserted 26-Sep-2037 23:58:20, got 26-Sep-2037 23:58:20, OK
Inserted 15-Aug-1994 07:44:57, got 15-Aug-1994 07:44:57, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 13-Sep-2022 23:14:04, got 13-Sep-2022 23:14:04, OK
Inserted 23-Mar-2031 15:29:59, got 23-Mar-2031 15:29:59, OK
Inserted 27-Dec-1983 21:15:25, got 27-Dec-1983 21:15:25, OK
Inserted 29-Feb-2032 06:55:03, got 29-Feb-2032 06:55:03, OK
Inserted 17-Jan-2019 10:58:15, got 17-Jan-2019 10:58:15, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 09-Oct-2031 21:33:34, got 09-Oct-2031 21:33:34, OK
Inserted 21-Aug-2023 20:31:46, got 21-Aug-2023 20:31:46, OK
Inserted 20-Sep-1992 16:30:21, got 20-Sep-1992 16:30:21, OK
Inserted 11-Nov-1970 10:00:13, got 11-Nov-1970 09:00:13 <-------- FAIL
Inserted 24-Feb-1984 20:46:46, got 24-Feb-1984 20:46:46, OK
Inserted 19-May-2005 00:45:39, got 19-May-2005 00:45:39, OK
Inserted 22-Apr-1986 05:51:55, got 22-Apr-1986 05:51:55, OK
Inserted 10-Apr-1987 11:32:32, got 10-Apr-1987 11:32:32, OK
Inserted 28-May-2016 15:43:58, got 28-May-2016 15:43:58, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 11-Feb-2033 01:03:55, got 11-Feb-2033 01:03:55, OK
Inserted 10-Jul-2031 19:50:26, got 10-Jul-2031 19:50:26, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 23-Nov-1982 04:12:36, got 23-Nov-1982 04:12:36, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 11-May-2009 21:59:43, got 11-May-2009 21:59:43, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 07-Jun-2007 01:11:58, got 07-Jun-2007 01:11:58, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 14-Mar-2002 06:34:51, got 14-Mar-2002 06:34:51, OK
Inserted 09-Nov-2009 06:40:03, got 09-Nov-2009 06:40:03, OK
Inserted 30-Jul-2037 06:55:44, got 30-Jul-2037 06:55:44, OK
Inserted 26-Nov-2030 21:14:53, got 26-Nov-2030 21:14:53, OK
Inserted 05-Sep-1996 15:14:24, got 05-Sep-1996 15:14:24, OK
Inserted 07-Apr-1980 11:34:26, got 07-Apr-1980 11:34:26, OK
Inserted 02-Jan-2037 18:55:00, got 02-Jan-2037 18:55:00, OK
Inserted 14-Mar-1977 15:07:19, got 14-Mar-1977 15:07:19, OK
Inserted 16-Oct-1995 01:51:15, got 16-Oct-1995 01:51:15, OK
Inserted 04-Aug-1990 06:50:10, got 04-Aug-1990 06:50:10, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 23-May-2021 16:00:23, got 23-May-2021 16:00:23, OK
Inserted 17-Aug-1982 02:21:05, got 17-Aug-1982 02:21:05, OK
Inserted 27-Aug-2013 20:52:49, got 27-Aug-2013 20:52:49, OK
Inserted 13-Dec-2027 14:10:48, got 13-Dec-2027 14:10:48, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 29-Jun-2025 02:53:11, got 29-Jun-2025 02:53:11, OK
Inserted 24-Jul-2031 23:54:31, got 24-Jul-2031 23:54:31, OK
Inserted 15-Mar-1971 21:08:49, got 15-Mar-1971 20:08:49 <-------- FAIL
Inserted 27-Apr-1981 21:35:54, got 27-Apr-1981 21:35:54, OK
Inserted 22-Dec-2008 19:00:03, got 22-Dec-2008 19:00:03, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 11-Feb-1986 09:25:28, got 11-Feb-1986 09:25:28, OK
Inserted 24-Mar-1971 12:46:15, got 24-Mar-1971 11:46:15 <-------- FAIL
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 29-Sep-2016 04:41:02, got 29-Sep-2016 04:41:02, OK
Inserted 03-Dec-2000 09:58:00, got 03-Dec-2000 09:58:00, OK
Inserted 10-Dec-1991 18:08:10, got 10-Dec-1991 18:08:10, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 29-Oct-1999 04:17:24, got 29-Oct-1999 04:17:24, OK
Inserted 21-Oct-1988 14:15:16, got 21-Oct-1988 14:15:16, OK
Inserted 27-May-2022 04:21:34, got 27-May-2022 04:21:34, OK
Inserted 16-Oct-1982 05:25:39, got 16-Oct-1982 05:25:39, OK
Inserted 19-Nov-1998 14:57:54, got 19-Nov-1998 14:57:54, OK
Inserted 29-Jun-1975 10:06:11, got 29-Jun-1975 10:06:11, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 02-Jul-1996 03:08:55, got 02-Jul-1996 03:08:55, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 17-Jan-2016 17:46:01, got 17-Jan-2016 17:46:01, OK
Inserted 28-Feb-1993 16:57:25, got 28-Feb-1993 16:57:25, OK
Inserted 21-Dec-1977 16:54:30, got 21-Dec-1977 16:54:30, OK
Inserted 05-Mar-2003 12:58:52, got 05-Mar-2003 12:58:52, OK
Inserted 03-Jul-2023 17:06:21, got 03-Jul-2023 17:06:21, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK
Inserted 06-Aug-1981 02:57:40, got 06-Aug-1981 02:57:40, OK
Inserted 17-Nov-1983 23:55:58, got 17-Nov-1983 23:55:58, OK
Inserted 17-Mar-1999 22:40:16, got 17-Mar-1999 22:40:16, OK
Inserted 04-Oct-2023 18:55:54, got 04-Oct-2023 18:55:54, OK
Inserted 28-Feb-1991 09:52:00, got 28-Feb-1991 09:52:00, OK
Inserted 13-Dec-1901 20:45:52, got 13-Dec-1901 20:45:52, OK

The only thing I can see is that they’re near the Unix epoch, but why would that cause it to be exactly 1 hour out…? The latest version of the code is up on Github. The underlying C code is in oci_types.c.

Anyway, at least this illustrates the value of soak-testing with randomly generated data – I had never experienced this issue “in the wild”, not has it been reported.

Update: Fixed! Was a double-application of localtime. I never noticed it because at the company I was at when I wrote this, there was a policy of all machines everywhere in the world being in GMT all year round! The epoch thing was a red herring. I suppose the moral of the story is make sure your random data is really random…

Posted in Ocaml, OCIML, Oracle | 5 Comments