Category Archives: Tools

Project List

The main purpose of this blog is to keep track of my various projects and to document them. The end result would be a long list of projects with all the associated metadata which can then be summarised to make browsing them easier. The Project List is a meta project that does just that. Each project on my website gets a file named project.php associated with it that contains the meta information. They are then gathered by the Project List and can be sorted, filtered, and displayed online. I’ve also added static pages, blogs, and talks to the lists of resources. You can view the Project List to read more it. This was tied in to a major update of my website, as I had to create new preview images and new styles to match the new way of showing the projects. This will probably evolve further over time, but for now it’s fine.

Screenshot

Project List, showing some projects
Project List, showing some projects

Update: Alarm clock

In the past few weeks I’ve been giving my website a facelift and updating many of the projects that were looking a little less presentable than the rest. One of the biggest changes was to the Alarm clock (Live page). Previously the clock had to be set in the source code (which was easy enough for me to do) and set using a time interval. I updated the code so that any user could set their own alarms and use a fixed time rather than an interval. At the same time I improved the general layout of the page and am now rather pleased with how it looks.

Before and after

Alarm clock before the update
Alarm clock before the update
Alarm clock after the update
Alarm clock after the update

Update: Painter

Recently I wanted to make some pixel art for a mobile version of my website (which is very different to anything I’ve done before, and more confusing than I thought it would be.) To do that I decided to use the Painter tool (Live page) I’d written a few years ago. Unfortunately the tool no longer worked, for reasons that I’m still trying to debug. (Possible to cross-domain restrictions when using the full path, but more likely due to the limits of the GET request.) The old method used PHP to generate the image, and the HTML DOM to manage the editing, using an HTTP request when the image needed to be generated. It’s not hard to see why this is inefficient, so once the canvas became available I decided to transition from PHP to JavaScript and the canvas.

The painter, after a facelift and going from PHP to JavaScript.
The painter, after a facelift and going from PHP to JavaScript.

Given my previous experience with the canvas it didn’t take long to code up a new version of the painter, and the biggest challenge was extending the scope of the line paintbrush to create lines at any angles, not just vertical or horizontal. The updated look is more sleek and professional, the performance is better, and there are fewer failure modes. I’ve kept the legacy version for interest, but I have no plans on using that again. I am also considering adding two new featuers in the future: allowing the user to save their image to an online database, and allowing the user to upload an image so they can edit existing images pixel by pixel.

Project management

Here’s something special for the 100th post! It’s project I use to manage projects.

This project was made to manage other projects. Each project has its own metadata which is parsed and arranged to give a useful summary of what the project does, how it was developed, and what I learned during development.

Links

Blog
GitHub repository

Overview

Each project has a python script that summarises the metadata, including the GitHub repository, the relevant links, a short description of the project, and details of implementation. Summaries are made for the Projects WordPress blog, and special links made for the PHP pages for projects that are hosted on my website. The scripts are organised so that they iterate over all projects (currently 83 and growing) at once, allowing a summary of all projects to be formulated.

This project management will eventually be extended to include icons, images, and other useful information. So far this has mostly be used to allow me to catalogue my existing projects in my spare time, which has taken ~15 months, but now that is nearly complete my needs will change, so I will add new functionalities.

Challenges

Challenge: This project needs to be versatile enough to handle a lot metadata about many projects.
Solution: In order to accommodate so many projects I had to create a new top level directory in my user space which is arranged hierarchically by category to contain all the projects. The choice to use python is motivated by the need to have a project management system that is versatile enough to handle a wide divrersity of projects with different metadata. (Resolved.)
This is how the sausage is made...
This is how the sausage is made…

i (greasemonkey)

One of the projects I wrote was an indico uri shortening service, called i (or indigo, depending on how ugly “i” looks in context) but I wanted to go a step further. So I write a greasemonkey script so that I could get to indico pages without even loading the main page. This takes up very few pixels on the screen, making it much faster to get to the meeting pages.

Links

indigo page
GitHub repository

Overview

This is a rather simple script. It simply redirects (in a new tab) to the i page, which redirects to indico.

Screenshot

Screenshot of a tiny implementation of i
Screenshot of a tiny implementation of i

LXR tidy

When I was working on the CMS experiment I used their LXR a lot when browsing code. The interface was fairly basic and would return a poorly formatted (and unsemantic) HTML output, so I write a greasemonkey script to make the results easier to read. Within a couple of weeks of writing this code the developers improved the output, so this project never got any wider use.

Links

CMSSW LXR home
GitHub repository

Overview

The output is parsed and carefully separated into different results. These are then wrapped in more pleasant looking HTML, colour coded based on the file type (with characters for the colour blind) and the main modules are summarised to give an idea of the main users of a given piece of code.

Challenges

Challenge: The default output is terrible. Just terrible.
Solution: The default output was not even semantically correct (from what I remember there were no <html> tags) so I had a hard time parsing the DOM properly. Special character sequences such as << appearing in C++ code didn’t make things much easier. (Resolved.)

Screenshots

Screenshot of LXR output before the tidy script is applied
Screenshot of LXR output before the tidy script is applied
Screenshot of LXR output before the tidy script is applied
Screenshot of LXR output before the tidy script is applied
Screenshot of LXR output after the tidy script is applied
Screenshot of LXR output after the tidy script is applied
Screenshot of LXR output after the tidy script is applied
Screenshot of LXR output after the tidy script is applied

LGBT map

One of my more far reaching projects is the LGBT CERN group. It is a diverse group with people from across the world, and one of the issues that we care about is safety in different nations. This project keeps track of progress in different nations.

Links

Live page (Google Maps)
Live page (Colourblind safe)
GitHub repository

Overview

The information is stored in xml files which are then read and used to create maps. The user can step forward and backward through the history, or let it autoplay. There are two versions of the page, once which uses Google Maps and one which has a custom map which is more colourblind friendly.

There is scope to extend this project for other uses, and I also have maps showing the state of the EU and its history.

This project is currently unfinished and needs some further cleaning up of code when time allows. This was based on Google maps API experience from the Railway tickets project.

Challenges

Challenge: Finding vectors for the national borders was not easy.
Solution: After much searching I found some useful vectors for the national borders. They are used here with two caveats: they are not my intectual property, and they are not small in size. As a result I am not too keen to share this project with the wider world. (Resolved)
Challenge: Making a colourblind friendly Google Map is not easy.
Solution: I wanted to make the map colourblind friendly, and the simplest way to do this was to use striped fills. This is not a satisfctory solution using Google Maps, so I made a second page where is is a reasonable solution. This meant that I had make a page that writes maps from scratch, including panning and zooming. This was easier than expected and may lead to other map-based projects in the future. (Resolved)
Challenge: This project used striped fills in the polygons.
Solution: One of the more difficult parts of this project was developing an algorithm that intersected multiple polygons in a consistent and sensible way. After much experimentation and development, this was achieved. This used knowledge developed on a previous and unfinihsed project that creates city skyline graphics. (Resolved)
Challenge: I needed an xml reader.
Solution: There are xml readers available in Javascript, but I chose to use my previously developed library that I had used for the Marble Hornets project. This should probably be replaced with a json solution later. (Resolved)

Screenshots

LGBT map screenshot - Google Maps version
LGBT map screenshot – Google Maps version
LGBT map screenshot - Colourblind safe version
LGBT map screenshot – Colourblind safe version

ScramBuildEggs

This project parses a build log and gives more intuitive output.

Links

GitHub repository

Overview

This is a script that parses the build log created by the CMSSW command “scram build” (hence the name!) It takes the output, wraps it to a more readable width, splits errors by type and colours them, and also gives more useful output. It’s not complete yet, but it is already rather useful.

Screenshot

ScramBuildEggs screenshot
ScramBuildEggs screenshot

Symbol browser

This is a simple tool to allow the user to quickly copy a string of symbols.

Links

Live page
GitHub repository

Overview

This is a mixture of Javascript and the flash based ZeroClipboard. The user clicks on the symbols they want and copy the output with a single click.

Challenges

Challenge: The project required making ZeroClipboard work.
Solution: Making ZeroClipboard work for single clikcks is not trivial for several DOM elements, so it now only works for a single DOM element. (Resolved)
Challenge: The list of symbols needs to be updated simply and easily.
Solution: Symbols are arranged in groups with special functions to manage them, so symbols can be added in a single line. (Resolved)

Screenshot

Symbol browser
Symbol browser

Recycling old solutions

A few years ago I found myself in need of a simple program to make pixel art. At the time I had access to the PHP suite of graphical libraries, and the ability to manipulate the HTML DOM, so armed with these tools I put together a simple tool that would allow me to create small images online, which I called the Painter. It was not particularly efficient, but it got the job done, and did so for free. (Now that the HTML canvas is supported pretty much everywhere that tools needs to be rewritten for the latest generation of browser technology.) During the development of this tool I found myself wanting to solve the following problem: Given a two dimensional rectangular array of squares, how can I find all squares that reside within a given boundary? It’s a fairly straightforward problem, but one that requires keeping track of two lists of squares, and one that should scale, as far as possible, with \(n^2\) or lower. After some head scratching and experimenting with a few lines of code I found the solution I needed, tweaked it a bit, and was fairly impressed with its elegance.

Filling complex shapes with the Painter
Filling complex shapes with the Painter

Then a few years later I was talking with my current boss and she said something along the lines of “You shouldn’t take on these extra projects, they take up your time and distract you.” It’s a fair point that with over 50 coding projects, and about a dozen more nebulous and time consuming projects, I have a lot on my plate, but on the other hand they serve some very useful purposes that I can’t get elsewhere. One of the most useful contributions they give is that they are often the first testing ground for solving problems. The lessons I learn in Javascript, PHP, and python are often directly applicable to my physics work.

The CMS Preshower in real life (Image: Anna Pantelia/CERN)
The CMS Preshower in real life (Image: Anna Pantelia/CERN)

A few months ago I found myself having to deal with the CMS Preshower system. I wanted to make a map of the system to that I could visualise how it looked in three dimensions. The only things I really knew as that it was arranged in a grid of square components (each square component subsequently arranged in strips) and that I had a tool where if I could access a single square I could access all of its neighbours. Since the physical extent of the preshower system gave me an obvious boundary I found myself with a very familiar sounding problem… Without even needing to look up the code from the Painter tool I wrote down the solution in a few lines and in about 10 minutes. What may have taken me a few hours of framing the question, determining how to find the solution, and subsequently implement it was reduced to a trivially solved problem with a minimal footprint of CPU time and memory. This later lead on to the development of the \(\eta-\phi\) map, which is still under development and may eventually lead to more breakthroughs for the CMS experiment.

The end result, a preshower map!
The end result, a Preshower map!

I make it a point to not invest much time in a project unless it’s going to teach me a new skill or show me how a new technology or feature works. By exploring what was possible with a few rudimentary tools I was able to unwittingly give myself the solution to a real world problem that saved a lot of headaches and time. Whether the extra projects are worth the extra time is a different discussion entirely (I personally think they are, and also think that what I learn in physics programming today I can use in private enterprise tomorrow) and one that deserves some serious thought. For now I am happy that I have a huge pool of experience to draw on that only keeps growing in time.