Databases in 2023
From Lee Robinson's Substack. Man is he ever doing a good job of marketing himself.
I'm not sure how I feel about most of the databases and database technologies that he's on about. I'm peripherally aware of a lot of them but I've used very few. And this post does open my eyes to how many there are.
My approach to databases has always been:
- Can you get away with SQLite?
- Okay, well can you get away with regular ol MySQL?
And then from there, I try to put the application as close to the database as possible. Ideally on the same £6 DigitalOcean droplet that I've got everything else on.
The database providers that Lee's talking about, however, don't operate persistent connections to the apps querying them. Instead:
Web native: Browser data-fetching APIs (e.g. Web fetch) and protocols are eating the world. New databases use HTTP APIs or WebSockets, rather than opening direct connections to the database. This makes them compatible with all forms of compute (including the lighter runtime used in edge compute).
Surely that's a radically inefficient way of going about things. I thought the whole point of databases was quick access to stuff? TCP is sloooow. Take this fun diagram from Neon, who are building a protocol on top of Postgres for accessing data over Websockets:
Not too long ago, this diagram would have looked like this (where the parentheses indicate the boundary of the hardware running the code):
users <- HTTP -> (application <-> database)
Serverless has been commoditised to the extent that people don't want to think about databases anymore—I get that. It's on the vendors to make sure that accessing your data is snappy. But this feels like the same weird gaslighting around 2017 when create-react-app was king and we were jumping through hoops to get SEO to work with client-rendered applications. I must be missing something here.
I'm also a little suspicious of Robinson's claim that "developers don't want to dial the knobs." I think there's a very specific type of developer that he's talking about, here: 3-6 years in the industry, working at an agency or product startup, frontend-oriented, introduces themselves as "full-stack" at Valley meetups.
But one thing that working at the NHS has illuminated for me is the vast unaddressed demographic of developers building the systems that all of these other systems operate on. Systems engineers at any of the database startups that Lee links to—hell, systems engineers at Vercel!—care about dialing the knobs for sure!