On the german list someone came up with the eeePC. This is from size exactly what I am looking for and you can get them second hand at around 50 euro.
I had 3. One was my laptop (with external keyboard), one was my son's and the other was my wife's desktop (with external kbd/mouse. One broke and the other two are parked away since years.
The processing power is way too limited and the storage is awfully small. It was useable (but not exciting) when released, but newer stuff is more resource-greedy. At one debian upgrade I had to remove most package beforehand, and at the next step I gave up. My use is pretty basic, but a local "git clone" of a kernel tree, with external storage, took hours.
Sure you can ssh elsewhere, but I we can do that with the phone (and an external keyboard). Any local use, besides running mutt and an editor, makes no sense on it nowadays. Even building firmware for microcontrollers is waaay too slow.
/alessandro