LEDE project: prevent having to re-flash the latest nightly build every time you need a new package

I really admire the LEDE project and the reasons why they decided that it was necessary to fork openwrt and continue in a way that they saw fit. Unfortunately as they only recently started (somewhere about May 2016) there are no official releases yet and this means that there are only nightly builds.

This 'guide' will work for both openwrt and lede nightly builds. The reason is that every night the packages and dependencies get rebuilt and packages that rely on a specific kernel/kernel module version can not be installed at a later moment (when the new nightly build replaces the previous one online). To solve this we can make a copy of all packages for a given nightly build. In a nutshell:

1. Download the firmware file from the LEDE/openwrt site
2. Run the commands that download all packages from the LEDE/openwrt site
3. Install/upgrade to recent build
4. Run a local webserver
5. Modify the package repository URL's to match your local webserver

So lets get started!

Download the firmware file

This is easy and probably something you've done before. If not: check out the architecture of your router and try to find whether openwrt/LEDE already supports your router. If they don't you're out of luck for now. For this guide I'll be using the TP-Link Archer C7 which is contained in the ar71xx builds of both LEDE and openwrt. 

Run these commands that download all packages from the LEDE/openwrt site

wget -m --no-parent -e robots=off https://downloads.lede-project.org/snapshots/targets/ar71xx/generic/packages/

wget -m --no-parent -e robots=off http://downloads.lede-project.org/snapshots/packages/mips_24kc/base

wget -m --no-parent -e robots=off http://downloads.lede-project.org/snapshots/packages/mips_24kc/luci

wget -m --no-parent -e robots=off http://downloads.lede-project.org/snapshots/packages/mips_24kc/packages

wget -m --no-parent -e robots=off http://downloads.lede-project.org/snapshots/packages/mips_24kc/routing

wget -m --no-parent -e robots=off http://downloads.lede-project.org/snapshots/packages/mips_24kc/telephony

The commands above should download all the packages that LEDE references within it's "software distribution feeds". For openwrt all references to lede-project.org should be replaced with openwrt.org. If you already have a version of openwrt/LEDE installed you can just make sure that you wget every of the URL's in the list instead of the URL's I provided above. Checkout 'System > Software' => Configuration tab for a list of feeds used by the current firmware.

After downloading the feeds you might want to move the contents of the downloads.lede-project.org/snapshots folder to a new folder. I'm using /Users/flix/Documens/lede-snapshots here as the /Users folder can be shared with Docker containers by default if you're using macOS. Not sure about how permissive Linux is in this regard.

Install/upgrade to the downloaded build

After downloading the packages you can install the firmware. To be sure that your 'offline' package feeds actually will work I suggest not (re)installing any of your packages just yet. Just the firmware.

Run a local webserver

Of course, the goal of downloading the packages is to serve them at a later moment in time. Let's get it started:

docker run --name nginx -v /Users/flix/Documents/lede-snapshots:/usr/share/nginx/html:ro -d -p 8080:80 nginx

The above command will pull an nginx container from docker hub and tell it to serve your local folder (containing the packages) on port 8080 from the machine you're running on. Of course you can copy the files to your home server, NAS, raspberry pi or any other device that can run a web server permanently, but for my use case (occasional desire to try out a package) I find spinning up a docker container the easiest.

Modify the package feed URLs

So now we can just swap the downloads.lede-project.org/snapshots part with the IP address of our webserver. This is it! You can now install packages from your local package mirror! Good luck!

The HP Microserver Gen8 for EUR 200. Is it worth it?

So recently our Drobo at QwikSense (my other company) died. Which wasn't really a problem, except that it was storing our Time Machine backups for my computer and my colleagues' MacBooks. So I was looking for a reliable storage solution with room to spare for an extra disk and preferably with ECC memory.

In the NAS world you're either out of luck if you're looking for something at around €200. Most of the time this will give you an ARM device with HDD-slots. On top of that this locks you into the vendor firmware (and support) and most of the time building something yourself with the provided kernel/firmware sources is not enough to build a robust solution that gives you a 'simple' user interface that just does what it should.

Of course we could build something ourselves. Whack a Mini-ITX board in a dito casing and voilà: hard disks attached to your network. But I don't have the time to do this and the number of Mini-ITX boards that sports an ECC-capable CPU/RAM at a reasonable price is close to zero.

Our saviour

Enter the HP Gen8 Microserver. The model with a Celeron G1610T and 4GB ECC(!) RAM can currently be purchased in the Netherlands for €203 euros at the cheapest reliable internet store at the time of writing! Now the strange (and awesome) thing about this tiny server is that it has a CPU, memory and a motherboard that support ECC! Can you believe it? And as an extra, HP's ILO4 management solution is also present. You can actually turn this thing on remotely if it's switched off. You can forcefully reboot it if the kernel crashes. It has features that "real" servers have, but in a tiny and silent package that looks AWESOME.

Note that the image above was ripped from 3dnews.ru. Also note that this is not the exact same model that you can get for such a low price. My model did not come with an optical drive, but who needs that anyway?

Current setup

We still had some drives from the Drobo, so we put those in and added a 128GB SSD from Plextor to act as a caching drive. This confirmed my suspicion that the Drobo was in fact dead as a dodo. The drives were all okay! Because some of the drives that were used in the Drobo were WD Greens (and everybody is always crying about how WD Greens drop out of RAID arrays) I decided to WDIDLE3 them. The easiest way for me was to get UBCD, put it on a USB drive and boot the Microserver from that. Now I must stress that you should NOT connect ANY other drives than the ONE drive you want to flash (except for USB disks of course, they don't count). When booting from de USB drive, open "Disk Management" or something similar (sorry, I forgot what it actually was) and then select WDIDLE3 at the bottom of the list of tools. You might get a list with a lot of choices and a countdown. Let the countdown pass and wait for a DOS-like prompt. At the prompt enter the following command:

WDIDLE3.exe /s300 (for 5 minutes).

That's it. Power down the system and repeat for any WD Green drives you have lying around. Unraid time!

Unraid

Since I saw LinusTechTips' 7 Gamers 1 CPU build I was very interested in unRaid as a storage solution. While I'm still running my Netgear ReadyNAS 4 Ultra at home (running ReadyNAS OS6 while Netgear only provided updates on OS4) I have wanted to own a machine like this. Docker on the ReadyNAS wouldn't work (because I believe the kernel was too old, not sure if that is still the case since they migrated to Jessie in the latest firmware update) but on unRaid the sky is the limit as VM's, Docker and plugins can all be used out of the box. There is a very active community providing solutions to run Plex Media Server, Resilio Sync, CouchPotato, SabNZBD and many more tools for media management. As we use Resilio Sync at work I decided to install that. 

Note that the basic license for unRaid allows a maximum of 6 internal drives (== SATA) while the MicroServer only has 5 SATA ports (the fifth port is for the optical drive). The USB drive does not count towards the number of drives. Unless you're planning on adding a SAS card and an external storage enclosure you'll be fine with a basic license. And if you're not, you probably weren't planning on keeping the budget low anyway.

Photo from a review on tweakers.net
(dutch)

Power consumption

While I've read different thing about the Gen8 Micro I've not been able to measure the power consumption yet. Hopefully I'll be able to report on that soon.

Upgrades (and potential for win)

The Gen8 Microserver has a regular desktop-formfactor Intel CPU on board in socket 1155. While the CPU has a TDP of just 35 watts, many people have retrofitted their Gen8 with beefier CPU's. This is one of the cheapest ways to create a powerful, compact and mostly silent reliable server for a small company or for media streaming. Especially if you have a few drives lying around, this project (including CPU upgrade, SSD cache disk and unRaid license but excluding HDD's) would set you back around €420. Sounds like a sweet deal to me! But even without the CPU upgrade it's a great deal. Let me know what you think!

Energy measurement with Domoticz and Grafana. Yes, it's easy!

A while ago I started tracking the power and gas consumption at home. To do this I read the meters every sunday afternoon and this gave me some insight in the power consumption. Unfortunately this also means that you have the measurement resolution of a week, so you can't really change your habits and see the effect on power/gas consumption.

11 Months Later: About productivity and mental health as a software dev

Recently I was asked whether I could share the tips and tricks that I still implement from my post from about 11 months ago. At first, I wasn't even sure that I would be able to name things I've learned and managed to keep doing.

Then I started thinking: is it even important that I can name them? Do I have to be consciously aware of the improvements I've made? But maybe the more important thing to realise is: even if I gain a single improvement from reading a book, something tiny that I did not do before, it is a gain and it is worth it! On top of that, everyones situation is different. This is why the stuff that helped me might not necessarily work for you.

So in my opinion books like these should be used more like frameworks, toolboxes, but not as a manual or bible. Similar to how this post describes the concept of Scrum implementation:

The Scrum framework leaves different options and tactics to play the game, ways that are at any time adopted to the context and circumstances.

Even if you implement only a small subset of the possibilities that Scrum provides, you're still benefiting from it and you can choose to combine it with other tools within (or even outside) the Scrum framework to streamline (optimise, improve, whatever) your work(flow) even more. This is something that is explained based on examples and anecdotes in the book Scrum & XP from the trenches, which you can download for free!

In conclusion

While I don't want to sum up the things I do better because they might not work for you, I'll list the things I currently do and things that I wish I would be better at implementing. These are from Blueprint of a productive programmer as the other book (Becoming a better programmer) is basically a huge list of great advice that you should probably read every 4 weeks until you really think you're implementing > 10% of it :).

Things I have learned and am able to implement (at least partially):

• Chapter 2 from Blueprint of a productive programmer: 
• Minimise Distractions, Stay off Facebook (I actually uninstalled FB and WhatsApp from my phone)
• Avoid meetings (unless it's REALLY clear what needs to be discussed for both parties and other means of communication will NOT suffice)
• Commit to repository often (especially if you have trouble focussing because of stuff like A.D.D.)

Stuff that I really want to do better (but have forgotten to actually keep doing)

• Chapter 5 from Blueprint of a productive programmer:
• Eat the right food
• Take regular breaks
• Prevent or treat RSI

At the same time I found the book as a whole to give me a great insight in overall pitfalls in your daily routines as a programmer. If you're a programmer (and even if you're a pretty good programmer) I'm sure you will gain something from reading it :). Again, here are the books:

Becoming a better programmer by Pete Goodliffe:

O'Reilly Store, Amazon (Kindle)

The Blueprint for a Productive Programmer by Moshfegh Hamedani:

Amazon ebook

Tidying up the "server room"

I have a NAS, a UPS, a Powerline adapter, a Philips Hue Bridge, an OpenTherm Gateway, and an OpenPi (Raspberry Pi Compute Module breakout board) in the hallway closet. There also was a 2nd gen Airport Extreme, but it has been replaced with a less power-hungry and more performant alternative. But even then, it was an utter mess.

Unfortunately I have forgotten to make a 'before' picture, but I assure you, the empty space next to the NAS was filled with the items listed above. Wires everywhere. And even removing the Airport did not solve this.

Then I decided, everything with mounting holes has to hang on the wall. And sometimes it's these small things that make a big difference. Mounting a piece of wood to the wall and putting some screws in allowed me to reduce the cable clutter (most of the cables can now be tucked away behind the NAS) and because the devices aren't laying all across the place, power cables are now routed together in a bundle.

Now the only thing I have to do is wall mount the OpenPi. But I'll do that some other day. Even small improvements like this can make a great difference: it's so much easier to service a device without accidentally unplugging something else! Do yourself a favour and go after the low hanging fruit. You'll thank yourself later :)

So PalmOS is really dead now?

I've been enjoying me PalmOS devices for quite some time now. First as an organizer and multimedia device and later as part of my collection of obsolete technology. What I really liked about PalmOS was it's speed and 'simplicity' if I may call it that. Of course I'm looking at an of from the past, knowing that there are a lot of OSes for portable devices that have a lot of features that PalmOS never had.

In the past years I've had a Sony Clie PEG-N760C and a PEG-NR70V. Later I upgraded to the Tungsten T2, then the T3 and finally the LifeDrive (which I've upgraded to an 8GB CompactFlash card). Of all those devices, only the T3 and the LifeDrive are still with me (and in working order :D). Recently I acquired a Palm Tungsten T|X, which was one of the last devices that Palm made before switching over to WebOS on the Pre series of smartphones. I was surprised to find that this device actually has received a patch/update to support WPA2 and WPA Enterprise! That is friggin AWESOME! While the Tungsten T|X only has Wireless B built in (802.11b = 11mbps) it works, and the device probably wouldn't be able to handle much more throughput anyway. Because the digitiser was broken, I ordered a new one from eBay, and it was pretty easy to replace it. It seemed like I had a high-end PalmOS device with WiFi, Bluetooth and a "high res" screen.

This is how I got started on wanting to revive PalmOS, at least from a developers perspective. I wanted to write (or port) a JSON library and use it to interact with REST API's or something. Or a remote control for my home automation system. Or basically anything that I could code that did not exist yet for PalmOS. But I hit a few roadblocks.

Lack of documentation and community

There isn't a lot of 'publicly available' documentation on PalmOS development. While the SDK docs can be found in some of the deep dark corners of the web, and a development environment can be set up, there is just no active community at all. It seems like development has stopped for about 10 years (or more!) and all what is left is these obsolete devices that are not able to use most of their functionality. Considering the fact that the devices aren't sold anymore, there is no market for software development on PalmOS and you're not actually gaining anything other than personal satisfaction (most people have forgotten that PalmOS existed at all!) this seems reasonable.

SSL Certificate support is limited to SHA1

While some tools exist to convert CA root certificates to PDB files, the PalmOS does not support SHA256 certificates. This means that most of the HTTPS interwebs is not reachable from the browsers, and secure communication is basically impossible. Most companies that have a WPA2 Enterprise-secured WiFi-network use an SHA256 certificate and this makes using the Tungsten T|X on a corporate network (or country-wide WiFi-hotspot network like the Dutch Ziggo WiFiSpots) impossible. SHA256 has been used in at least one application that I know of, pssh2. Unfortunately re-using this code in an application and rewriting part of PalmOS are two different things.

So what can we still do with our legacy devices?

When these devices were still awesome, a lot of cool stuff has been made. And because the internet was not as big as it is today, most of these apps work offline! Over the next few weeks I'm going to show a few applications that I've used in the past decade and might be nice for you to re-live the PalmOS days again :).

Felix from the 2022 here:

The PalmDB community strives to host files for abandoned projects as well as ROMs for devices and other useful tools. Check it out here. They have a Discord as well! Also Dmitry Grinberg is hacking Palm OS like there is no tomorrow, and there is a guide on how you can get started with a toolchain for PalmOS development here. Of course there is a subreddit as well: /r/Palm. Have fun!

Unbricking the HM-10 BLE module on the OpenPi

I've been doing some home automation testing using the OpenPi because of it's small footprint, internal RTC and the cute case that was created for it. As a cherry on top, they also included an SRF/XRF module and a Bluetooth LE (HM-10) module so you can remotely open a shell using a Bluetooth-enabled device and an app.

My goal was to scan for devices in the neighborhood and determine whether me or my girlfriend is home so we can turn down the central heating. Using the information found in a document explaining the available AT commands I got no response to the scanning command. I decided to update the firmware of the HC-10. I can now safely say that I have confirmed the module in the OpenPI to be the TI CC2541. In the flashing manual the following message can be found:

I assumed that no firmware manufacturer could ever actually write firmware that could leave the device in a bricked state unless you provide firmware. What if the device isn't actually connected to a windows pc? Well, after issuing the software upgrade command that was exactly the situation I was in.

To save you the trouble, the tool that Huamao Technology provides does not work correctly using the workaround I did to flash the HM-10 on my OpenPi. Luckily the tool provided by the manufacturer of the bluetooth chip DOES work! So download the firmware from Huamao's site (HM-10, CC2541) and follow the next few steps to bring your HM-10 back from the dead.

Requirements:

• A Windows (virtual) machine to run the update tool on
• Raspbian or similar image on your OpenPi, SSH enabled
• A network connection between your Windows PC and OpenPi
• socat and minicom installed on your OpenPi (sudo apt-get install socat minicom -y)

Step 0: (if you did not brick the HM-10 yet but are feeling adventurous) set the HM-10 module to Software Upgrade mode:

ssh into your raspberry pi.

Run: minicom -s

A menu opens. You want to select "Serial Port Setup", Press 'A', replace ttyUSB0 with ttyS0 (assuming this is the device that your HM-10 is connected to), press Enter.

Setting E should be set to 9600 8N1. If this is not the case, press E, then C, then Enter.

If your screen looks somewhat like the screenshot above, press Enter. You should now see the minicom menu. Select Exit to open the session. You will not see what you're typing. If you prefer seeing what you type, press Ctrl + A, then Z, then E. This turns on the "Echo".

Then paste the following command to place the module into Software Upgrade mode:

AT+SBLUP 

After this your device will be in Upgrade mode. You can exit minicom by pressing Ctrl + A, Q and answering 'Yes'.

From this point on, the HC-10 module switches to 115200 baud for the software upgrade.

Step 1: Start Socat on the Raspberry Pi

To provide raw TCP access to the bluetooth modules serial interface, run the following command:

while true; do
  socat tcp-listen:54321,reuseaddr /dev/ttyS0,b115200,raw,echo=0
done

This command makes socat listen on port 54321, and if the server is stopped by a reset command, a new server is started.

Step 2: Install Eltima Serial to Ethernet connector (trial) on your Windows machine

Download the trial, install it and open the program that is installed on your machine.

Step 3: Create a connection to your socat instance on the pi:

This is what the setting should look like.
Replace the hostname with the IP of
your OpenPI.

- Choose 'create a client connection'
- Select an unused COM-port number (I chose COM3)
- Fill in the remote hostname (IP address of your OpenPI)
- Fill in the port number (54321, the socat listening port)
- Open 'Advanced Settings'
- Disable 'Connect to Remote end only when local virtual port is open' checkbox
- Enable 'On error retry to establish connection every' checkbox, and set its value to 100ms.
- Press 'Apply Changes'

Step 4: Install and open the TI SerialBootTool.

Step 5: Select the firmware file and serial port. 

After selecting the serial port, press the 'Open' button.

Step 6: Flash away! 

After flashing and verification (which may take a while, approx. 5 minutes?) you should be good to go!

This should be the result of your hard work!

Of course, because of the while loop, we started on the Pi it is probably best to close the session where the loop is run. This will also kill the loop. Otherwise, just restart the pi by unplugging the MicroUSB plug providing power to the device.

Other helpful and somewhat related HM-10 resources:

http://smoothieware.org/bluetooth-serial
https://github.com/WirelessThings/OpenPi/blob/master/gettingstarted.rst
http://openmicros.org/index.php/component/kunena/14-openpi/11317-bluetooth-module?Itemid=0
https://github.com/RedBearLab/CCLoader/blob/master/Arduino/CCLoader/CCLoader.ino
http://raspberrypi.stackexchange.com/questions/15382/raspberry-pi-used-as-a-cheap-serial-to-wlan-converter

So why don't Google and Azure have full IPv6 connectivity yet?

For my company I've been using different cloud providers in the past years. One of the things I noticed is that Google, Amazon and Azure currently don't offer IPv6 connectivity for virtual machines at all! The following is Google's statement at the time of writing:

All Compute Engine networks use the IPv4 protocol. Compute Engine currently does not support IPv6. However, Google is a major advocate of IPv6 and it is an important future direction.

Azure states something similar:

 Microsoft has played a leading role in helping customers to smoothly transition from IPv4 to IPv6 for the past several years. [...] The foundational work to enable IPv6 in the Azure environment is well underway. However, we are unable to share a date when IPv6 support will be generally available at this time.

So how is this even possible? I understand that there are technical challenges, but IPv6 has been around for years and these guys really are really big players in the cloud market!

So what ARE your options for native IPv6?
Well, basically you can use an Amazon load balancer with IPv6 support, create a tunnel for IPv6, or use another hacky solution, but for now I'm afraid native (and vendor supported) IPv6 is only available with smaller hosting companies, Softlayer and Linode being great examples of providers that do offer IPv6 with virtual machines. But if you require specific vendor services from Google, Amazon or Microsoft you'll be left without official IPv6 support for VM's. At least, for now.

Introducing the Epoch WebService for Low-Power IoT devices without RTC

Edit: the epoch service is down. There are so many alternative ways to do something like this. 

Today I was working on the OakPowerBoard project and I asked myself: why isn't there a web service that just allows you to get the current Unix Timestamp without having to parse loads of response data? Well, I couldn't find it! So I built it and deployed it to Azure! Until I find a shorter domain you can use the following URL as you see fit!

  http://tiwtieapp.azurewebsites.net/now?format=json

This will return the server time. No padding, no trimming, just the time! Neat, right? I've contacted the owner of epo.ch, but haven't had a response yet.

So why did I do this?
The OakPowerBoard firmware does HTTP requests anyway, so doing a request to this web service does not cost a lot of extra code. Implementing a bare-bones NTP client is probably a lot more work! And of course, nearly every internet-connected platform has examples for a simple HTTP GET request, this cannot be said for NTP. Enjoy!

Oak Power Board, Part 1

I've been meaning to do a small write-up on my latest project, and add-on board for DigiStump's Oak board; an ESP8266-ish-based mini board with an on-board LDO that is programmable with the Arduino IDE using Particle Cloud. The Oak was successfully funded using KickStarter and I got my board pretty fast! While the firmware is not yet finished I've been able to make a prototype and think about what I wanted to do with a cute Arduino-IDE-compatible-wifi-equipped-board like this. You can find the project on GitHub.

Creating a Power-consumption monitoring device

I've recently moved in with my girlfriend (who does awesome drawings, by the way!) and because this meant having to pay the electricity bills ourselves I wanted to know whether we would have to pay extra (for consuming more electricity and gas than anticipated) or that we'll receive money for consuming less of both. I was also interested in how much energy was consumed at a given time of day. This helps us decide which devices to keep plugged in in stand-by, and which devices to fully unplug. Because I'd heard of some Arduino-based projects using an TCRT5000 to do just this, I decided I wanted to design a board to do the same, but instead it would be cheaper, easier (both to use and to update, thanks to Particle OTA updates), and smaller than a regular Arduino.

Prototype

After some prototyping I decided on using an SSD1306 OLED display (128x64 pixels) that I has lying around at work. I hooked it up, and (fortunately!) someone had already written a library to use the SSD1306 on the ESP8266. As Particle handles WiFi connectivity that meant that the only thing I had to do was write code to calculate the time between pulses of the KWh-meter, and I should be good to go, right? So I decided to design a PCB to put the Oak, the TCRT5000 and the SSD1306 display on.

Designing the first board (also: the first mistake)

Because the prototype seemed to do what it should, I put a PCB design together and sent it to OSH-park. I received the boards pretty soon, and I was happy with them, until I tried to get measurements from the TCRT5000. I had forgotten to check whether we would be using the TCRT5000 input as analog or digital. Where the Arduino has a lot of ports to use (usually 6 or more analog, and a lot more of the digital kind), the Oak has only a single port that is able to do analogRead (port 0). Fortunately I had been using port 5, so I could add a bodge-wire and continue with my firmware.

 

The bodge-wire in question, redirecting the sense-pin connected
to the TCRT5000 to the only analog pin on the Oak

On a side note: I really like the board as it is. OSH-park is great and I would recommend it to anyone who wants to do a small prototype run. Compared to services I've used, the web-based interface is easy and shows renders of how the board will look. Every layer is separately rendered and I've been able to spot some mistakes before sending the board to fab. The purple color is funky, and of course that also counts for something, right?

Current stage of the project

The latest developments on the board have been that I'm currently not able to reliably attach the board to my KWh-meter. This means I can't write/test code at the moment, but I hope to be able to test this soon. If pulse detection works I can do some tests on other meters and find a way to semi-permanently attach the device to anyones (spinning disk type) KWh meter.

If you want to check it out, go to github. Both the board and the code files are there. I've opened a topic on the digistump boards for this project, so if you already have an account there, you can also contact me in the topic on the Projects board. If you have any suggestions, contact me or leave a message. Thanks for reading! 

Update on recent developments

Hi All! As you might (or might not) know, I have my own company (thisiswhytheinternetexists), but I also work at QwikSense, which is my other company that I co-founded. As QwikSense has some spare server capacity I'm able to use that for TIWTIE as hosting that doesn't need a lot of resources anyway. Unfortunately that is the reason that my site has been down for the past few days: QwikSense moved from Google Cloud to Microsoft Azure, and I didn't have the spare time to fix the site. Luckily, I found some time to do this today, and as you might be able to see, it's up and running again! Yay!