Helsinki

A few weeks ago I had the good fortune and pleasure of being able to visit Helsinki. I was there for the 102nd meeting of TC39, which was co-hosted by Mozilla this time. As someone interested in standards and potentially increasing the amount of standardization work I do, I thought this would be a particularly good meeting to dip my toes into.

Helsinki on kaunis kaupunki.

Helsinki is a beautiful city. Visiting in June, I was blessed with long daylight hours and much less rain than I had expected.

Suomi on iloinen maa

Finland is a happy country... according to the World Happiest Report, apparently the happiest country for the last 6 years. Exploring Helsinki I could totally see it. While it's extremely hard to extrapolate from a city to a whole country, I found Helsinki seemed to feel like a place where life would be comfortable in many ways. What I experienced was a city dense with public transit, filled with apartments and parks, and active recreation routes throughout. You saw Finns taking their children places in cargo bikes, people commuting by bike or public transport, and enjoying the many small shops throughout.

I got the vibe (and this is distinctly a vibes-based piece) that Finland due to its unique language has been invaded just a bit less than other countries by international giants... though I definitely saw a few Burger Kings.

Suomalainen Sauna on todella hyvä

Finnish Sauna is really good. I was really lucky; one of our Finnish Mozillian colleagues took a number of us from TC39 to Lonna, an adorable island a brief ferry ride away from Helsinki's market square. There we enjoyed for a couple of hours a wood fired Finnish sauna. I even managed to swim in the Baltic Sea at Lonna... twice!

Sauna is a huge part of Finnish culture -- it's included on UNESCO's Representative List of the Intangible Cultural Heritage of Humanity. Experiencing it on Lonna was wonderful, and I'm very glad that was my first experience. Emboldened by Lonna, a couple of days later I tried the sauna in my hotel, and the experience was starkly different. Where Lonna was hot, but felt like a warm embrace... the hotel sauna felt much more like being in an oven. I couldn't take the heat in the hotel sauna at all.

I took a little time and explored Suomenlinna while I was in Helsinki. It is an inhabited set of islands that were fortified into a defensive fortress by the Swedish and Russians. It was a fascinating outing — I gently regret not taking a guided tour of some sort. It was here where I was attacked or harried by so many birds. So many angry hungry birds.

Suomalainen on kaunis kieli

My wife has been learning German casually through Duolingo for a while. I think we both acknowledge that it's not going to teach you a language, but you get to sample the language in small chunks -- and you do pick things up, just not detail or grammar or culture. Nevertheless, the day I booked my flight to Finland I joined her on Duolingo, but for Finnish.

Finnish has a bit of a reputation for being challenging; it's not a relative of many languages (Wikipedia tells me that the three largest Finno-Ugric languages are Finnish, Estonian and Hungarian). I found learning Finnish to be a very mind-opening experience. While there's lots about Finnish that is hard (and I have barely scratched the surface: search for 'istahtaisinkohankaan' on the Wikipedia for Finnish to see the derivation of how that word means "I wonder if I should sit down for a while after all"), there's also lots about it that feels very regular, which as an English speaker is deeply unfamiliar. For example, pronunciation is quite regular, a huge change from English.

I didn't have high hopes for using my very weak Finnish skills in Finland, and I was largely correct there. I never spoke it to anyone. Still, it was neat to be able to decipher the signage; some time just reverse engineering some of the signage based on what I did know.

I told myself when I got back I'd likely move on to something else. I really enjoyed the brain expansion I got from ~70 days of Finnish, and figured that when I got back I'd try something else... I was thinking perhaps learning enough Hangul to sound out a food menu. But I have found myself returning to Finnish. It's still enjoyable to practice and learn. Who knows how long I will continue, of whether I will ever use it... but I am enjoying it now, so why wouldn't I keep at it.

Hyvä Suomi

Bravo Finland. I had a great time visiting, and you've made me want to return. Thanks for the wonderful time -- Apparently compliments are challenging, but truly, Thanks.

Living Through History

“ Men are apt to turn their eyes upon the past, as holding all that is worthy of contemplation, while they fail to take note of history “in the making,” or to grasp the importance of the transactions of their own day.”
— EVELYN S. SHUCKBURGH

(from here)

Scattered Thoughts on LLM Technology

In a sense, this is maybe part of the blog post I promised back in January 2023 but never wrote, because I couldn't ultimately wrap my head around LLMs enough to form coherent thoughts.

I am less skeptical of this than I used to be, and I find a number of arguments about it less compelling. For a longer, more compelling, less scattershot version of this post (not all of it, but some of it), please read The average AI criticism has gotten lazy, and that's dangerous by Danilo Campos.

  1. On the energy consumption case, I have to remind myself sometimes that I am not inherently against energy consumption. It must be done thoughtfully, and renewably, but energy use by itself is not disqualifying. Furthermore, there is going to be huge amounts of financial and competitive pressures for LLM technology to get leaner, using less compute.

    This is a contrast to blockchain, where in proof-of-work systems, resource consumption is the point. Efficiencies are automatically consumed by difficulty scaling, and so individual actors can find efficiencies and profit from them, the system as a whole cannot.

  2. On the training data problems: This is a legitimate concern; yet it strikes me that it is not impossible to overcome; Similar to how LLMs will have strong downward pressure on their compute and energy consumption, LLMs are experiencing downward pressure on their data requirements. While huge data and huge compute are required today, I think it's short sighted to assume the same will be true tomorrow. The idea of a small model that is specialized to mozilla-central is kind of appealing!

  3. It feels like there has to be a bit of a cost reckoning; If OpenAI isn't profitable (and I don't know that we know if that's true or not), we could see costs climb for the deployment of AI models. Spend some time playing with the OpenAI cost calculator, and it's pretty clear that at scale it's already reasonably expensive to use. I think we will see three trends here: 1) Companies that are just thin wrappers of OpenAI’s APIs will slowly disappear; why go third party when OpenAI will serve you (for cheaper almost certainly). 2) You’ll see at least two hardware startups blow up on their ability to dramatically reduce cost for running these open models. E.g. play with Groq demo. Partially driven by cost, I also think

  4. The “just throw a chatbot on X” model of deployment we’re seeing now will fade away; a lot more usage will be LLM-as-API; I expect this also means API focused models. Ones where output is always JSON, and they are task trained more than general purpose. I’ll bet we even see input stop being “prompts”, as the technology adapts to traditional programming over time.

  5. This space is going to change remarkably over the next few years; we're in for a period of dramatic change here. I've no idea how it's going to go, but it's going to look wildly different in five years I suspect. I work at Mozilla, and the Mozilla Foundation has been thinking about this a lot.

  6. I worry so much about bandwagoning, and bad choices driven by FOMO; I think so many of the "rub-generative-ai-on-it" projects are gross, and many of them undermine the value proposition of the organizations deploying them. Yet, I am increasingly convinced that I can't put a blanket over my head and hope this all blows over... it's not blowing over.

For more, be sure to also read Campos' What if Bill Gates is right about AI.

Books of Early 2024

Let's talk a bit about some books I read early this year (some are also waiting for a longer blog post too).

I'm loathe to publicly call out bad books, but I will say, one book not represented here is one I paid good money for, and hated. Everything about it says I should have loved it; a well-loved author, genre and subject a lovely mishmash that should have worked for me, and yet less than a third of the way into the book I had to stop. I haven't hated a book like this in a long time. All the reasons why I should have loved it made the pain of it being bad all the more painful.

Three Men in A Boat

I am gently shocked I didn't write a blog post about this book previously; this was actually my second read through. Published in 1889, it's a humour book which holds up ridiculously well for something which is 135 years old. I first discovered it after it was mentioned/tied gently into Connie Willis' To Say Nothing of the Dog, so it's likely my first read through of Three Men in a Boat was roughly 2018.

In theory it is the story of three young men taking a break from their lives to take a vacation by boat up the river Thames, and it is that; however, a good chunk of the book however is humorous digressions. One of which is very prescient reads on our contemporary character:

To go back to the carved-oak question, they must have had very fair notions of the artistic and the beautiful, our great-great-grandfathers.  Why, all our art treasures of to-day are only the dug-up commonplaces of three or four hundred years ago.  I wonder if there is real intrinsic beauty in the old soup-plates, beer-mugs, and candle-snuffers that we prize so now, or if it is only the halo of age glowing around them that gives them their charms in our eyes.  The “old blue” that we hang about our walls as ornaments were the common every-day household utensils of a few centuries ago; and the pink shepherds and the yellow shepherdesses that we hand round now for all our friends to gush over, and pretend they understand, were the unvalued mantel-ornaments that the mother of the eighteenth century would have given the baby to suck when he cried.

Will it be the same in the future?  Will the prized treasures of to-day always be the cheap trifles of the day before?  Will rows of our willow-pattern dinner-plates be ranged above the chimneypieces of the great in the years 2000 and odd?  Will the white cups with the gold rim and the beautiful gold flower inside (species unknown), that our Sarah Janes now break in sheer light-heartedness of spirit, be carefully mended, and stood upon a bracket, and dusted only by the lady of the house?

Others instead poke fun at the manner and mores of the day; visit the Project Gutenberg ebook, and read from "Have you ever been in a house where there are a couple courting?".

Or perhaps if you're in for the fantastic story of a ripe cheese, read from "For other breakfast things, George suggested eggs and bacon".

Persuasion

When I got sick recently, I discovered all I wanted was Austen. So we watched Sense and Sensibility, Pride and Prejudice , Mansfield Park (Grim) and I read Persuasion. First time reading it, though I was familiar with the story beats, having watched an adaption last year.

Persuasion is an interesting book; tension wise, it is a slow burn, building to its crescendo only in the final chapters. It's a good love story, told well, in a world that feels lived in.

I was struck however by the characters. Last year I read Middlemarch (which I apparently never wrote about?? Something I find gently shocking considering how highly I now regard it!), and what I find having read the two near each other is how much more alive the characters in Middlemarch feel -- their emotions more clear, their strains more communicated.

I am very happy I read Persuasion, but I think I preferred Pride and Prejudice when I read it a few years ago.

The Very Secret Society of Irregular Witches

This was just a very cozy story of found family in a gentle magical world just behind the veil of our own. It was warm and fuzzy (with a small amount of sex, including one ??? moment for me), and it was just the right thing for a cold winters day.