Category Archives: Data analysis

Update: Trigger game

In recent weeks I’ve been working a lot on the Trigger game (Live page.) These update have included an overhaul of the style, addition of new pages, rewriting of the “Spy mode”, adding new particles, tweaking the graphics, and many more changes behind the scenes. The code was significantly refactored to make it easier to extend and understand, as now this has becomes a collaborative project. The game has been tested on a few schools and shown to be a good success with children (and adults) and it seems to have a bright future. I also added sounds, music, and a simple music player.

Screenshots

Updated event display
Updated event display
Showing the various modes (some in development) with a garish colour scheme.
Showing the various modes (some in development) with a garish colour scheme.
Updated Spy mode, analysing the data.
Updated Spy mode, analysing the data.

Work in progress: Trigger game

Like the Science Shift Simulator, this is a game that emerged as a subset of the aDetector project. The player has to save events that match the criteria given, just like a trigger does in real life. The results are then combined across all the players in a given team to determine the final score. The scores can be combined across experiments to make a “discovery”. This is still in development, which will continue over the net week and months.

Links

Live page
Spy page
GitHub repository

Overview

This is another cooperative multiplayer game aimed at showing the public (especially high school pupils) how particle physics research actually takes place. Any number of players can take part and they are split into “Team ATLAS” and “Team CMS”. The score of each team is determined by the performance of the players on each “shift” they take at the trigger, and the final scores are combined for the discovery of the Higgs boson. There is also a “spy” mode where people can see the events as they are submitted.

Challenges

Challenge: This project needs an attractive, fast, and realistic detector model.
Solution: Having already developed a decent detector model for the aDetector project, I simply used a two dimensional version for this project. I then split the detector finely in \(\eta\) and \(\phi\) to make interactions between the particles and detector easier. The aesthetics went through a few iterations before settling on the final design. However further optimisations and aesthetic changes are anticipated as development continues. (Resolved, to be revisited.)
Challenge: This game puts a bit of a strain on my server.
Solution: My web space uses a shared server, so sometimes many HTTP requests from the same client looks like a Denial Of Service (DOS) attack, resulting in a throttling of the requests. There are two main strategies to overcome this. The first option is to bundle up several requests into one request, reducing the total number of requests, and the load on the server. This solution has not been implemented yet. The second option is to change the server settings. I do not have access to these, but as development continues I intend to move to a different server that can handle so many very small requests. (To be revisited.)
Challenge: This game needs cross platform and cross device support.
Solution: This game was initially intended to be played with an iPad, but I did not have an iPad for testing. On the day of the release of the game I had to tweak the settings so the response times were slower with respect to mouse events to make it easier to play on a tablet device. These settings are trivially changed to allow multiple device support. (Resolved.)
Challenge: The game should be repsonsive to the inputs of the user.
Solution: Initially the game did not confirm success when a user clicked on the screen and this lead to confusion. As a result I had to add a big green “tick” for success and a big red “cross” for failure to inform the user of the status of the event. (Resolved.)
Challenge: The game needed an animated histogram for the final result.
Solution: I’ve made histogras before, including histograms that update themselves on the aDetector project, but until now I had not animated a histogram. This was a bit tricky as I had to call events which were using the this keyword, so the histogram object had to be stored as a global variable because, although I’d like to, I couldn’t use this to refer to the histgoram. Javascript can be frustating like that. (Resolved.)

Screenshots

I don’t normally put lots of screenshots up, but I’m quite proud of the asethetics here, so here are the three main screens of the main game:

The front page of the game.
The front page of the game.
The transition page of the game.
The transition page of the game.
The game in progress.
The game in progress.

The design went through a few iterations before settling on the current choice:

Design 1
Design 1
Design 2
Design 2
Design 3
Design 3

aDetector

One of the projects I have wanted to develop for a long time is a browser based particle physics experiment simulator. Such a project would generate events using Monte Carlo methods and simuate their interactions with the detector. This was made partly as an educational aid, partly as a challenge to myself, and partly because at the time I was feeling some frustration with the lack of real analysis in my job. As expected for a Javascript based CPU intensive appplication, this reaches the limits of what is possible with current technology quite rapidly.

Links

Live page V1
Live page V2
Live page V3
Live page V4
Live page V5
GitHub repository

Overview

The model for the detector is saved in an xml file which is loaded using the same methods developed in the Marble Hornets project. Particle data are currently stored in Javascript source files, but will eventually use xml as well. The particle interactions are simulated first by choosing a process (eg \(e^+e^-\to q\bar{q}\)) and then decaying the particles. Jets are formed by popping \(q-\bar{q}\) pairs out of the vacuum while phase space allows and then arranging the resulting pairs in hadrons. Bound states are then decayed according to their Particle Data Group (PDG) branching fractions, with phase space decays used. The paths of the particles are propagated through the detector using a stepwise helix propagation. Energy depositions in the detector are estimated, according to the characteristic properties of the detector components. A list of particles is then compiled based on the particles produced and these can be used to reconstruct parent particles.

The user has access to several controls to interact with the application. They can choose how to view the detector, using Cartesian coordinates and two Euler angles (with the roll axis suppressed.) The most expensive parts of the process are the generation of the event displays and the generation of the particle table. By default these are only updated after a certain interval, to allow the user to accumulate a significant number of events without being slowed down by the graphics. To save time the detector itself is rendered once in a cutaway view, and the particle tracks are overlaid on the saved image. Eventually the user will be able to get a full event display, including the detector response to the particle with glowing detector components etc.

The user has access to collections of particles, including electrons, muons, pions, kaons, photons, and protons. From these they can construct other particles, making selections as they do so. Once they have made parent particles they can then plot kinematic variables including mass, momentum, transverse moment, and helicity angle. This should, in principle, allow students to learn how to recreate particles and how to separate signal from background effectively.

Given the large amount of information available the user has access to a number of tabs which can can be collapsed out of view. This allows the user to run the application with the expensive canvas and DOM updates, and thus collect many more events.

This is still a work in progress, with reconstruction of particle being the next main priority. Eventually the user would be able to load their favourite detector geometry and beam conditions, then perform their analysis, saving the output in xml files and possible being able to upload these to a server. This would allow users to act as “players” with “physics campaigns”, including the SPS experiments, HERA experiments, B factories, LEP experiments, and LHC experiments. This is, of course, a very ambitious goal, and one which has been ongoing for over a year at this point.

See other posts tagged with aDetector.

Challenges

Challenge: A sophisticated model for the detector was needed.
Solution: The detector is split up by subdetector, with each subdetector having its own characteristic responses to different particles. The detector is split up in cylindrical coordinates, \((
ho,\eta,\phi)\), with each subdetector also being split into modules. Individual modules then react the particles for reconstruction purposes. Thus with a few key parameters even a sophisticated model can be stored in a few variables that can be tuned quickly and easily. (Resolved.)
Challenge: The detector shold have a three dimensional view that the user can control.
Solution: The detector is drawn using a wireframe with transparent panels. This is a method I developed in 2009 for a now defunct PHP generated SVG based visualisation of the BaBar electromagnetic calorimeter, which I used to show the absorbed dose as a function of detector region and time. The drawing algorithm is not perfect, as panels are drawn in order from furthest from the user to closest. This is sufficient for most purposes, but occasionally panels will intersect causing strange artefacts. Eventually this should be replaced with a much more stable, robust, and fast implementation, such as in three.js. (Resolved, to be revisited.)
Challenge: Particles should be modeled realistically and physically correctly.
Solution: Particles are modelled with the most important parameters (mass, charge, decay modes etc) taken from the PDG. Their kinematic properties are modeled using special four vector classes, and decay products “inherit” from their parents in a consistent manner. Particles at the highest layer of the tree are assigned their four momenta, and then their decay products are decayed, inheriting the boost and production vertex from their parents. This is repeated recursively until all unstable particles are decayed. So far this does not take spin into account, as everything is decayed using a phase space model. Particles with large widths have their lineshape modeled using a Breit-Wigner shape. As a result, particle have realistic looking jets and four momentum is conserved. This required the development of special libraries to handle these decays and kinematic constraints. (Resolved, to be revisited.)
Challenge: Particles decay trees must be traversed consistently.
Solution: This is harder than it sounds! Every time an event is generated, particles are recursively decayed for as long as phase space allows. The particles must then be traversed and dispalyed in the table, in a consistent manner. Ensuring that all particles are listed hierarchally without missing anything out takes some care. (Resolved.)
Challenge: Particles lists had to be prepared for the user.
Solution: The user has access to a handful of “building blocks” to play with. These are taken from the (generated) list of particles per event and filtered by the particle type. Further lists can be created or filtered from these lists, and parent particles can reconstructed from combining lists. This means having to provide special classes to handle the particles and ensure that no particles are reconstructed recursively (leading to infinite loops.) Apart from using reconstructed particles instead of generated particles, this has been resolved. (Resolved, to be revisited.)
Challenge: The user needs to be able to make histograms.
Solution: I had made histgorams for other projects, including the Reflections and Box plotter projects, so this was mostly fairly easy to do. Even so, being able to create new histograms of arbitrary scales and variables on the fly meant that this histogram class had to be more robust than previous projects. (Resolved.)

Screenshot

Screenshot of aDetector V5
Screenshot of aDetector V5

LGBT map

One of my more far reaching projects is the LGBT CERN group. It is a diverse group with people from across the world, and one of the issues that we care about is safety in different nations. This project keeps track of progress in different nations.

Links

Live page (Google Maps)
Live page (Colourblind safe)
GitHub repository

Overview

The information is stored in xml files which are then read and used to create maps. The user can step forward and backward through the history, or let it autoplay. There are two versions of the page, once which uses Google Maps and one which has a custom map which is more colourblind friendly.

There is scope to extend this project for other uses, and I also have maps showing the state of the EU and its history.

This project is currently unfinished and needs some further cleaning up of code when time allows. This was based on Google maps API experience from the Railway tickets project.

Challenges

Challenge: Finding vectors for the national borders was not easy.
Solution: After much searching I found some useful vectors for the national borders. They are used here with two caveats: they are not my intectual property, and they are not small in size. As a result I am not too keen to share this project with the wider world. (Resolved)
Challenge: Making a colourblind friendly Google Map is not easy.
Solution: I wanted to make the map colourblind friendly, and the simplest way to do this was to use striped fills. This is not a satisfctory solution using Google Maps, so I made a second page where is is a reasonable solution. This meant that I had make a page that writes maps from scratch, including panning and zooming. This was easier than expected and may lead to other map-based projects in the future. (Resolved)
Challenge: This project used striped fills in the polygons.
Solution: One of the more difficult parts of this project was developing an algorithm that intersected multiple polygons in a consistent and sensible way. After much experimentation and development, this was achieved. This used knowledge developed on a previous and unfinihsed project that creates city skyline graphics. (Resolved)
Challenge: I needed an xml reader.
Solution: There are xml readers available in Javascript, but I chose to use my previously developed library that I had used for the Marble Hornets project. This should probably be replaced with a json solution later. (Resolved)

Screenshots

LGBT map screenshot - Google Maps version
LGBT map screenshot – Google Maps version
LGBT map screenshot - Colourblind safe version
LGBT map screenshot – Colourblind safe version

Recycling old solutions

A few years ago I found myself in need of a simple program to make pixel art. At the time I had access to the PHP suite of graphical libraries, and the ability to manipulate the HTML DOM, so armed with these tools I put together a simple tool that would allow me to create small images online, which I called the Painter. It was not particularly efficient, but it got the job done, and did so for free. (Now that the HTML canvas is supported pretty much everywhere that tools needs to be rewritten for the latest generation of browser technology.) During the development of this tool I found myself wanting to solve the following problem: Given a two dimensional rectangular array of squares, how can I find all squares that reside within a given boundary? It’s a fairly straightforward problem, but one that requires keeping track of two lists of squares, and one that should scale, as far as possible, with \(n^2\) or lower. After some head scratching and experimenting with a few lines of code I found the solution I needed, tweaked it a bit, and was fairly impressed with its elegance.

Filling complex shapes with the Painter
Filling complex shapes with the Painter

Then a few years later I was talking with my current boss and she said something along the lines of “You shouldn’t take on these extra projects, they take up your time and distract you.” It’s a fair point that with over 50 coding projects, and about a dozen more nebulous and time consuming projects, I have a lot on my plate, but on the other hand they serve some very useful purposes that I can’t get elsewhere. One of the most useful contributions they give is that they are often the first testing ground for solving problems. The lessons I learn in Javascript, PHP, and python are often directly applicable to my physics work.

The CMS Preshower in real life (Image: Anna Pantelia/CERN)
The CMS Preshower in real life (Image: Anna Pantelia/CERN)

A few months ago I found myself having to deal with the CMS Preshower system. I wanted to make a map of the system to that I could visualise how it looked in three dimensions. The only things I really knew as that it was arranged in a grid of square components (each square component subsequently arranged in strips) and that I had a tool where if I could access a single square I could access all of its neighbours. Since the physical extent of the preshower system gave me an obvious boundary I found myself with a very familiar sounding problem… Without even needing to look up the code from the Painter tool I wrote down the solution in a few lines and in about 10 minutes. What may have taken me a few hours of framing the question, determining how to find the solution, and subsequently implement it was reduced to a trivially solved problem with a minimal footprint of CPU time and memory. This later lead on to the development of the \(\eta-\phi\) map, which is still under development and may eventually lead to more breakthroughs for the CMS experiment.

The end result, a preshower map!
The end result, a Preshower map!

I make it a point to not invest much time in a project unless it’s going to teach me a new skill or show me how a new technology or feature works. By exploring what was possible with a few rudimentary tools I was able to unwittingly give myself the solution to a real world problem that saved a lot of headaches and time. Whether the extra projects are worth the extra time is a different discussion entirely (I personally think they are, and also think that what I learn in physics programming today I can use in private enterprise tomorrow) and one that deserves some serious thought. For now I am happy that I have a huge pool of experience to draw on that only keeps growing in time.

A universe simulator

In the world of particle physics it’s possible to simulate (our best approximation of) the fabric of reality. The bad news is that this comes at a cost. Nearly all software is made in-house by scientists. Many of them are excellent programmers, but many aren’t. The documentation is often missing steps and there are sometimes dependecies of outdated packages. To make matters worse the documentation is often sparse and pointing to the wrong resources.

Today I’m trying to use two packages called pythia and Delphes, which are used to simulate particle collisions and the respective detector responses. So far I’ve had to visit the pythia page at least four times (including outdated pages that no longer exist) to find the correct location of the package. Once downloaded and installed, it turns out it requires some additional packages to run the parts I need. That means going down another rabbit warren of pages and outdated links to find out how to get FastJet and HepMC.

On one hand keeping all these packages separate and linking to each home page makes things more versatile and faster to develop. On the other hand having a single recipe or even a single installer can work wonders and get people using these projects more quickly and easily. That’s what the CMS experiment does, and it gets people past the fisrt hurdle. All you need to do is a simple cmsrel <name> and then cmsenv and you have a fully working, internally consistent copy of the CMS software infrastructure. Each step that gets added to the process is a hurdle between the user and the end result, which often leads to frustration to the point where people simply give up trying to make things works. There have been several times in the past where I’ve made the decision that continuing to pursue a given route is just pointless. That’s why whenever I make a package I try to give simple all-in-one instructions to make things as easy as possible.

In any case this blog is supposed to be a useful repository of knowledge about the things I make and do, so here’s the link to the recipe: Physics cheatsheets.

Marriage data scrubbing

This project was the first step in a larger project that scrubbed data from pages for analysis.

Links

Live page
GitHub repository
Wikipedia page

Overview

This page takes input from Wikipedia (although at the moment this is extracted manually from the main page) and then produces a C script which summarises the data for further analysis. ROOT is then used to make a plot showing the data. The idea behind the project was to make data scrubbing easier and to give me practice in data scrubbing. If I ever get sufficient time I intend to create some resources to make data scrubbing easier and make sharing of publically accessible data simpler. The the data are taken from an outdated Wikipedia page which is missing many sources.

Challenges

Challenge: The data needs to be scrubbed from the page easily.
Solution: To save time I manually copied the data from Wikipedia for offline analysis, but this should be fixed to read the data online. (Resolved, to be revisited)
Challenge: The output needs to be some kind of source file, for example C or python.
Solution: The source file I chose was a ROOT macro for convenience. Ideally the user should be able to choose what output they want (including xml) to best suit their needs. (Resolved, to be revisited)

Sample output

Anlaysis of marriage data
Anlaysis of marriage data

CERN Open Data (CMS)

In 2014 the CMS experiment at CERN released their 2010 data to the public for analysis. As a quick exercise I decided to analyse the dimuon mass spectrum to show that this could be done in a reasonable amount of time.

Links

Live page
GitHub repository

Overview

The input file is the 2010 Run B comma separated values file. The python script then produces mass spectra for same sign and opposite sign mass spectra and zooms in on the interesting regions.

Challenges

Challenge: The main challenge was that this project was made as quickly as possible.
Solution: This project uses python and existing ROOT libraries for the maximal development speed. The other data format available was using CMSSW and a virtual machine. In principle using CMSSW should be straightforward, but I decided against using this because the software was already four years old and support would be minimal or non-existant, even to current CMS physicsts. (Resolved)

Sample output

Charmonia dimuon spectra
Charmonia dimuon spectra
Full dimuon spectra
Full dimuon spectra
Bottomonium dimuon spectra
Bottomonium dimuon spectra

ROOT colors

This project creates an infographic showing the ROOT colors and how they relate to each other.

Links

ROOT page
GitHub repository

Overview

Physicists use ROOT to make plots and it’s often useful to be able to easily browse the color space. ROOT provides a color wheel, but I find the rectangular display very useful as well. In principle, colors in the same column should suit each other, which makes the rectangular display more useful than the color wheel.

Sample output

Rectangular array of ROOT colors
Rectangular array of ROOT colors

That “Aha!” moment in C++

One of my responsibilities as a CMS physicist is developing C++ and python code using the framework, CMSSW. The coding standards are very high for CMSSW, with strict checking and treating warnings as errors, so even the slightest lapse in safety causes compile errors. I just came across a build error because I tried to pass a const instance of class as an argument to a method and using a non const method of that class, giving the error message:

error: passing ‘const PixelMatchSParameters’ as ‘this’ argument of ‘float PixelMatchSParameters::sPhi1B()’ discards qualifiers

It had been a while since I’d seen this error, so I had to look it up and when it did, and I realised the problem a smile spread across my face. There’s something beautiful about how strongly cast C is, and how strict C++ forces you to be when passing around objects. When the code is clean and safe it’s something I can be proud of, and that’s one of the reasons I love C++. (On the other hand there are times when C++ makes me want to pull my hair out, but they are often the result of poorly documented code, or working with someone else’s perverse design choices.)

I’ve been developing in C and C++ for about a decade now, but so far I have not created my own standalone project from scratch before without building upon some other framework. Perhaps it’s time I did.