koldfront

Plugging a memory leak #haskell #munin

This graph tells a story:

I have been experimenting with writing Haskell applications to run websites for a while. So far I have a website for Feedbase, a blog engine with NNTP-interface, a small wiki, and a (primarily NNTP) ActivePub server.

Annoyingly the webservers have been leaking memory, to the tune of gigabytes per week. And I haven't been able to figure out why.

I'm a Haskell novice (yes, still) so I have been looking at stuff like "do I use foldl" and switching to foldl' and stuff like that I could find by searching for "Haskell" and "memory leak".

At one point I even extracted the pages from this websites' log and redid all the requests locally on my laptop - but I was unable to reproduce the memory usage ballooning that way.

At one point somebody figured out that the Spock Haskell web framework has a leak, so I switched to Twain. My applications still leaking, though.

Earlier this week, while sitting on a long train ride, I realized that one of the differences between my server setup and the local test setup on the laptop was that the Haskell applications are proxied behind Apache.

My code, especially in Sixpence - the wiki, is quite simple, so I was at a loss to figure out where the leak was, but then I thought "How about those middleware modules I'm using?" - and I tried turning all of them off in the wiki and half in the blog-engine.

The result is what the graph shows!

My guess currently is that it's the Gzip and Brotli middleware plugins I was using that caused the leaking when behind Apache. Furthermore, testing with curl it turns out that they aren't necessary, as Apache handles the compression.

So now I have removed the middleware modules - can't wait to check the Munin graph in a couple of weeks!