Mia Farrow (78).
A small TCP server #programming #python #haskell
Yesterday at work I needed access to the output of the locate(1)
command remotely.
I wrote a little daemon that I set up as a service on the file server
that I want to run locate on from another machine.
As I was at work, I implemented it in Python - the lingua franca there, and I was, after all, on company time/money.
It looked like this:
#!/usr/bin/env python3
# located.py - a small tcp server that listens on port 10811 for
# strings to locate, and returns located strings.
import socketserver
import time
import subprocess
class LocateDHandler(socketserver.BaseRequestHandler):
def handle(self):
client_ip, client_port = self.client_address
print("Connection from {}: ".format(client_ip), flush=True, end="")
if client_ip not in ("127.0.0.1"):
print("Not talking to you.")
return
search_for = self.request.recv(1024).decode("utf8").strip()
if "'" in search_for:
print("Not searching for '{}'.".format(search_for))
return
try:
print("Searching for '{}' ... ".format(search_for), flush=True, end="")
start = time.perf_counter()
found = subprocess.check_output(["/usr/bin/locate", search_for])
end = time.perf_counter()
print("found {} hits in {:0.2f}s".format(len(found.splitlines()), end-start))
self.request.sendall(found)
except subprocess.CalledProcessError as e:
print(e)
pass
if __name__ == "__main__":
socketserver.TCPServer.allow_reuse_address = True
with socketserver.ForkingTCPServer(("0.0.0.0", 10811), LocateDHandler) as server:
server.serve_forever()
The port number is ascii l = 108 followed by ascii o = 111, but with a digit cut off, to stay within the allowed port numbers :-)
I was quite happy with finding the socketserver module and the
relative ease it was to whip this together. The biggest problem I had
was when searching for documentation online, a lot of it referred to
SocketServer, which was in an older version of Python.
You'll notice that it restricts access to a set of IP-addresses - in
real life (work) the list is different. It also doesn't to the search if
there is a ' in the search term, even though I assume that
subprocess handles escaping for me.
The only flourish is logging how much time it takes, and logging on one line, flushing each part, making it more exciting to follow "live".
I'm not thrilled about the hardcoded length in the .recv() call, but I
decided it was "good enough".
When I got home, I reimplemented it in Haskell - without looking at the Python code (I would have to go to another computer to fetch it, to much faff).
This is what I came up with:
-- located.hs - a small tcp server that listens on port 10811 for
-- strings to locate, and returns located strings.
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Network.Simple.TCP
import Network.SockAddr (showSockAddrBS)
import Data.Text as T (pack, strip, unpack, elem)
import Data.Text.IO as T (putStr, putStrLn)
import Data.Text.Encoding as T (decodeUtf8)
import Data.ByteString.Char8 as BS (lines)
import System.IO (hFlush, stdout)
import System.Process.ByteString (readProcessWithExitCode)
import System.Clock
import Formatting
import Formatting.Clock
main :: IO ()
main = do
serve (Host "0.0.0.0") "10811" $ \(connectionSocket, remoteAddr) -> do
let remoteIP = T.decodeUtf8 (showSockAddrBS remoteAddr)
T.putStr ("Connection from " <> remoteIP <> ": ")
hFlush stdout
case remoteIP of
"127.0.0.1" -> T.putStr ""
_ -> error "No thank you."
searchFor <- recv connectionSocket 1024
case searchFor of
Just searchBytes -> do
let search = strip (decodeUtf8 searchBytes)
case T.elem '\'' search of
True -> error ("Won't search for \"" ++ T.unpack search ++ "\"")
False -> T.putStr ("Searching for '" <> search <> "' ... ")
hFlush stdout
start <- getTime Monotonic
(_, result, _) <- readProcessWithExitCode "/usr/bin/locate" [T.unpack search] ""
end <- getTime Monotonic
T.putStrLn ("found " <> T.pack (show (length (BS.lines result))) <> " results in " <> sformat timeSpecs start end)
send connectionSocket result
Nothing -> T.putStrLn "nothing."
It's quite similar, and a little longer.
I dislike having to convert to and from String in a couple of places,
because that's what the libraries return/expect, i.e.
showSockAddressBS and readProcessWithExitCode - and in general there
is too much faff to use Data.Text. It would be nicer if I could just
import Data.Text.putStrLn as putStrLn, even though the prelude has
the String version of putStrLn.
In the first iteration I converted the bytes coming back from locate's
stdout to Text, and then I converted them to bytes before sending
them. That failed when I hit file names with invalid utf-8 encoding, and
the explicit conversions made it easy for me to realize that I could
skip the converting and just treat the result as ByteString, as soon
as I found the lines function for ByteStrings.
When I made that change to the Haskell version, I went back and made the same change to the Python version, improving its robustness to rogue filenames.
While the concatenation of strings in the Haskell code is a little
clunky, at least it doesn't fall into Python's "there are too many ways
to format strings" trap (+? .format()? f""?).
Another thing I dislike in the Python version is the awkward idiom if __name__ == "__main__": at the bottom. That's just an ugly hack
compared to having the call to serve at the top.
Also, that allow_reuse_address defaults to False in Python feels
silly.
When started up the Python version uses 10MB of memory before serving anything, and around 11MB after serving a search with 1.8M hits in 1.89s. On disk the Python version takes up 1.4KB.
Similarly the Haskell version uses 4MB of memory before serving anything, and around 259MB after serving the search with 1.8M hits in 1.86s. On disk the compiled Haskell binary takes up 5.5MB (stripped).
Note that the timings are how long it takes to run locate, so it doesn't say anything about the languages.