I got the idea to make a small application to save web links from an IRC channel and list them on a web page.

IRC bot

The first part of the project was putting together a bot to sit in IRC and save links that were posted. In the past I made some simple IRC bots with Python in the past, there are lot of examples on the internet that use Socket to build things from scratch, but I decided this time to use the IRC module to get things going faster and take advantage of a tested stable library.

I was able to start with the Testbot example and because my bot and discard most of the Testbot class for everything except the handling public messages, private messages, DCC messages all being unnecessary for what I wanted.

With the bot connecting to IRC and joining a channel I then pass public messages to urlextract to check if the message contains a valid url, and save if it does. Initially for testing I just printed this to the console along with the IRC nick and the a timestamp.

I had a couple options for how to store the links, something like a plain text file or adding them directly into an html file and possibly just opening that directly in a browser, or a full fledged database. I decided to go with a sqlite database since I had worked with it before and since it wouldn't really be storing that much information (just a single table for the links) and didn't take up as much overhead as running a database server with more functions that I wouldn't be using anyways. Only one application can write to a sqlite database at a time but multiple applications can read from the database so for my project that was fine.

Lastly I switched from hardcoding the IRC server and channel information to having the script check for environmental variables with the information since I had an idea that I wanted to run this in a container later.

Web App

For the second part of the project I needed something to get the information out of the sqlite database and display somehow. Flask seemed like a good straight forward choice, since I really only needed a single route at first and I had used it for a couple projects in the past.

Getting started I setup a new script with a single Flask route that pulled all entries from the sqlite database and using the Jinja2 templating  it was straight forward to display them in on the page with a simple loop.  

Listing a large number of links on one page would make things pretty impractical and for the initial setup I was using SELECT * from links; to pull everything out of the database at once which is obviously a bad idea to leave in. I changed my SQL statement to select using DESCENDING, LIMIT and OFFSET to return the 20 most recent entries.

The simplest way I could think of to allow scrolling through the paginated results was to add a new route for / which returned redirect to /1 then knowing that any request would included a integer variable I used that for a page number, multiplying the OFFSET and then subtracting 20 in my SQL select, so the first page would start with row 0 and the 2nd page with row 20 etc. I added a next_page and previous_page variable to set < > links in the template to make requests for the appropriate page number, and doing a check to make sure that it never pointed to anything less than 1.

I didn't want to use an external library like Bootstrap that would need to be updated frequently or link to it online, or javascript. I did make use of nth-child in CSS to add some alternating background color for odd and even rows. Overall it doesn't look very impressive but is readable and plenty fast.

Containerization

I knew from the beginning that I wanted to run this in a container. Initially I looked at trying to run both processes in a single container either using a shell script to launch the irc bot and the flask app or using subprocess to start the irc bot from the flask app, but in the end decided the "Docker" way (and simplest way) was to use a separate container for each part.

For the IRC bot I used the Python's Alpine container as the base, and only needed to add a few lines to clone my repository in the container, install the requirements with pip and then start my script on launch in my Dockerfile.

Flask is pretty clear that it's built in web server is only suitable for testing, but despite this there are a lot of examples out there that don't mention this when they talk about setting up a Flask application in a container. After looking at options for a WGSI server I decided to use tiangolo/meinheld-gunicorn-flask as the base for my Dockerfile. Meinheld and Gunicorn were already configured in the container for a generic application so all I had to do was make sure I set the correct environmental variable to point Gunicorn to my application and it was ready to run outside of a testing.

Conclusion

There are still some changes I might make, maybe adding a route for displaying only links from a specific IRC nick, and one drawback of using a single repository for two separate containers is right now,  some unnecessary requirements are being installed in each container. Breaking the two parts into separate repositories would fix this, but even with the bloat the containers are both still pretty small thanks to Alpine, and this is just a quick project.

My biggest take away is really how many resources there are out there to get things built up quick, whether it's a Python Module or a container on Docker Hub. Thanks to all of these resources I was able to stand up my simple site really quick. Of course it's never a good idea to build on someone else's work (container or module) that you don't have any understanding of how it works.

The main project repository is available here and my Dockerfiles in a separate repository here.