koldfront

Hosts sending 2 or 6 NULL bytes #net

Recently, like the last month or so, my server has been receiving packets from various, seemingly arbitrary, hosts, containing either 2 or 6 NULL bytes.

They hit mostly port 22 (ssh), 53 (dns), 80 (http), 443 (https) and imaps (993). I only have a very limited number of ports open in the router, so they might be hitting more ports.

Looking at them with ngrep(8), it looks like this:

$ sudo ngrep -x -q '^\x00\x00*$'
interface: enp4s0 (192.168.1.0/255.255.255.0)
filter: ((ip || ip6) || (vlan && (ip || ip6)))
match: ^\x00\x00*$
 
T 88.208.41.159:42560 -> 192.168.1.101:22 [S] #32
  00 00                                                 ..
 
T 78.140.142.55:44230 -> 192.168.1.101:80 [S] #41
  00 00                                                 ..
 
T 78.140.141.247:47152 -> 192.168.1.101:53 [S] #46
  00 00                                                 ..
 
T 5.11.81.197:48256 -> 192.168.1.101:443 [S] #62
  00 00                                                 ..
 
T 54.36.150.116:41070 -> 192.168.1.101:443 [R] #404
  00 00 00 00 00 00                                     ......

I'm not quite sure to make of it. The sources seem to change, sometimes they're mostly from Japan, sometimes from China, sometimes from AWS, other times from various hosting companies.

Anyone know what this is?

 (0 comments)

rdiff-backup backport to Debian 11 (buster) #debian

A new version of rdiff-backup, 1.3.3, has been added to Debian unstable.

When that is used to backup to a machine running Debian 11 (buster), you get errors, because it has version 1.2.8.

As a work-around, I have backported librsync-dev and rdiff-backup from unstable to Debian 11 (buster), packages are in my local repository. Use at your own peril.

 (0 comments)

Making subprocess.Popen in Python 3 play nice with "elaborate" output #programming #python

docker-compose produces "elaborate" output when run, making use of carriage returns and ANSI CSI codes to move the cursor about.

A wrapper script written in Python 3 tried to handle the output - printing a '.' for each line usually, and actually printing the lines if given a --verbose option.

Unfortunately the output with --verbose got mangled, which really annoyed me. So I tried to find a solution.

Here is a small script that does some output, mimicking the stuff that docker-compose outputs. Note that we want the lines printed as they are produced, and not all at once at the end of the script.

command.py:

#!/usr/bin/python3
 
from time import sleep
 
print("1", flush=True)
sleep(1)
print("2", flush=True)
sleep(1)
print("3", flush=True)
sleep(1)
 
print("\x1b[2A\x1b[K\r", end="", flush=True)
sleep(1)
print("2 - let's go\r", end="", flush=True)
sleep(1)
print("2 - lookin' good\r", end="", flush=True)
sleep(1)
print("\x1b[1A\x1b[K\r", end="", flush=True)
sleep(1)
print("1 - abba\r", end="", flush=True)
sleep(1)
print("\x1b[2B\r", end="", flush=True)
sleep(1)
print("\r3 - flappa\r", end="", flush=True)
sleep(1)
print("\x1b[1A\x1b[K\r", end="", flush=True)
sleep(1)
print("2 - done\r", end="", flush=True)
sleep(1)
print("\x1b[1A\x1b[K\r", end="", flush=True)
sleep(1)
print("1 - done\r", end="", flush=True)
sleep(1)
print("\x1b[2B\r", end="", flush=True)
sleep(1)
print("3 - done\x1b[K\r", end="", flush=True)
sleep(1)
print("\nFinito", flush=True)

Try running it:

The original code wrapping it, was like this (slightly simplified):

runner_orig.py:

#!/usr/bin/python3
 
import subprocess
 
 
def run_command():
    p = subprocess.Popen("./command.py",
                         stdout=subprocess.PIPE,
                         stderr=subprocess.STDOUT,
                         universal_newlines=True)  # this converts \r into \n #fail
 
    for line in iter(p.stdout.readline, ""):
        yield line, p.poll()
 
    yield "", p.wait()
 
 
for l, rc in run_command():
    print(l, end="", flush=True)

If you run this, you'll see how the output it mangled:

So why does this happen? Well, the first culprit is universal_newlines=True. That means that any combination of carriage return and/or line feed is interpreted as and converted to a line feed. Uh-oh, definitely not what we want, if the fancy output it to be reproduced as intended.

If set to False, the situation improves, carriage returns are no longer converted. Unfortunately the p.stdout.readline() function only interprets line feeds as end of line and not carriage returns, so all the fancy stuff piles up, and is only shown at the very end. Not what we want.

Looking at the documentation of open() reveals that it has an option called newline, which is used to control how universal newlines are handled, and that if set to '' it actually does what we want: carriage returns are not converted, and they are recognized and line endings.

Unfortunately the documentation of subprocess.Popen() only documents universal_newlines taking two values, True and False, and '' doesn't work. I tried.

Fortunately Klaus had a tip - if you use os.dup(), you can open the file handle again, and - hey - now I can give it the option to open() we need, newline=''.

Lo and behold, it works!

runner_fixed.py:

#!/usr/bin/python3
 
import subprocess
import os
 
def run_command():
    p = subprocess.Popen("./command.py",
                         stdout=subprocess.PIPE,
                         stderr=subprocess.STDOUT,
                         universal_newlines=False)  # \r goes through
 
    nice_stdout = open(os.dup(p.stdout.fileno()), newline='')  # re-open to get \r recognized as new line
    for line in nice_stdout:
        yield line, p.poll()
 
    yield "", p.wait()
 
 
for l, rc in run_command():
    print(l, end="", flush=True)

It is kind of kludgy to have to do that, but it does work:

I wonder why subprocess.Popen() in Python 3 does not have the same options for handling newlines as open() has. It feels like open() has moved on (to the newline parameter), but subprocess.Popen() hasn't followed yet.

 (0 comments)

SanDisk Ultra Flair 64GB USB flash drive #hardware

USB flash drive Useful consumer review: The SanDisk Ultra Flair 64GB USB flash drive I bought last summer survived less than 1 year of rattling around in my bag.

Basically it has been on a keyring with a Yubikey Neo the whole time, and it just stopped working. As in nothing shows up in syslog or dmesg, and the USB device gets very hot quickly when inserted into a USB-port.

I tried prying it apart, to see if something was shorting and that was why it didn't work, but even inserting just the "core" of the device, without the casing yields nothing registered by the computer and a very hot device.

While it worked, it worked fine - but less than a year until catastrophical failure, without any warning, I give it 1 star.

So, what should I get instead? The Samsung BAR Plus Titan 128GB seems to have nice specs (300 MB/s).

 (0 comments)

Server upgraded to Debian 11 (buster) #debian

Debian 10 (buster) logo All in all it sort of ok.

Problems:

 (0 comments)

MIME support for inline images #email

I was about to send an image inline in an email using the venerable MIME standard, when I remembered that many clients don't handle that very well.

So I created a test email to see how they do.

The email is very simple - some text, an inline image, and some more text. Here is a copy, if you want to test your favourite client.

My email client shows it like this:

Gnus displaying my test email

Here are the results:

Client Shows image inline Shows text after image Screenshot
Gmail ❌ at bottom image
Office 365 Outlook ❌ at top❌ hidden as attachment image
Outlook iPhone ❌ hidden as attachment image
Protonmail ❌ as attachment at bottom image
Tutanota ❌ as attachment at top❌ hidden as attachment image
K9 Android ❌ as attachment at bottom image
Gnus image
Thunderbird image

So what I remembered was not wrong. Most email clients suck at MIME.

The Outlook app on the iPhone gets an extra half mark subtracted for showing an exceedingly ridiculous amount of vertical white space.

Which is not surprising, the relevant RFC is from 1996.

 (0 comments)

Installing Converse with Apache and ejabberd

Converse logo I banged my head against this for a while, so here is the short summary on how to get Converse going on a (Debian) server running Apache and ejabberd.

ejabberd is configured to have ejabberd_http_ws on "/websocket" on port 5280. This was already so in my setup.

I downloaded converse.min.css and converse.min.js from cdn.conversejs.org and put them in a folder called xmpp on my webserver. I also had to download a bunch of fonts: baumans.ttf, muli.ttf, fa-solid-900.{woff2,woff,ttf} and fa-regular-400.{woff2,woff,ttf}, which I put in xmpp/webfonts/.

Then I created xmpp/index.html, which simply contains:

<!DOCTYPE html>
<html lang="en">
  <head>
    <title>Converse - koldfront</title>
    <link rel="stylesheet" type="text/css" media="screen" href="converse.min.css">
    <script src="converse.min.js" charset="utf-8"></script>
  </head>
  <body>
    <script>
      converse.initialize({
          websocket_url: 'wss://koldfront.dk/websocket/',
          view_mode: 'fullscreen'
      });
    </script>
  </body>
</html>

Now, in Apache we need to include the proxy_wstunnel module, so run sudo a2enmod proxy_wstunnel. And in the configuration for the webserver, we need to tell it to proxy the /websocket URL to ejabberd:

    SSLProxyEngine on
    ProxyPass "/websocket/" "wss://koldfront.dk:5280/websocket/"
    ProxyPassReverse "/websocket/" "wss://koldfront.dk:5280/websocket/"

And then I was able to login.

I haven't figured out why image uploads don't work (they do from Conversations (Android) and jabber.el (Emacs)), but otherwise Converse seems to work.

 (0 comments)

Archive... Search... Keywords...
Sorø art museum

Tomorrow

Irma (133).

Den lille havfrue (106).

Sunday

Linux (28).

Monday

Colour television starts in Denmark (52).

Wednesday

Prins Nikolai William Alexander Frederik (20).

Software Freedom Day (15).

2019-08-29

Annisette (Savage Rose) (71).

0.0163 s
webcustodian@koldfront.dk