Posted on

How do programmers work?

Ever wondered how programmers know how to do everything on a computer? Well here’s an example.

I have a problem:  My computer doesn’t automatically pause the music I’m listening to when I leave my desk – so lets work out how to fix that. It seems like a simple thing to do, after all when I’m sitting at my desk all I need to do is move my mouse and press the pause button.

So how are we going to approach this? Lets first break down the task. Most computer systems do two things, they wait for an event and when it comes they trigger an action, so in our case our two tasks are:

  1. Detect when I’m away from my computer
  2. Trigger the music player to pause.

First step

If we are lucky then someone has already done this work for us and therefore our first step is to find out if it exists – our friend here is the internet, we’ll call him “Google”.

Me: "Does my music player stop playing when I leave my desk?"
Google: "Hi! We've found 2.9 million results! (we know you'll be interested in hearing that your search took 0.64 seconds to complete - you're welcome!) So.... Maybe this would help.... Maybe this would help....Maybe this would help... Maybe this would help... Maybe this would help... Maybe this would help... Maybe this would help... Maybe this would help... Maybe this would help... Maybe this would help... Next page."
Me: "No helpful results. Damn it.".

Second Step

No-one else has figured out how to do this, so our next step id to figure out if it’s even possible. We first want to check whether we can pause our music player without the mouse – if you thought we should check how to detect when I’m away from my computer first, you’d be wrong – there’s no point doing that if we can’t trigger a pause without using the mouse. So now we draw on one of three things:

  1. Our experience: we’ve done it before we know how to do it again.
  2. The documentation: The people who wrote the software know how to do it and they’ll tell us.
  3. The internet: Our friend will know what to do because someone else told him, and now they can tell us.

If you’re in number one, great! Get coding! Unfortunately the reality is there are too many pieces of software, written in too many different ways, for too many different systems that chances are you won’t have done it before. So onward to number two, but where’s the documentation? Many, many, many years ago in preInternetian times documentation may have come in a box in what was called a book, or, if you were lucky, on cartridge/floppy/CD/DVD. Nowadays all documentation is online – go directly to number three.

Me: "How do I pause my music player without my mouse?" 
Google: "Hi! We've found 4.1 million results and it just took 0.69 seconds, (I know, we're awesome!). And you lucky thing, We've picked this fantastic advert just for you - 'No hands? try  "
Me: "Ah look, the documentation for my music player. "

So you’ve found the documentation – it will be one of several things:

  • Too short
  • Too technical
  • Too long
  • Just right

Back to google, search properly:

Me: "How do I pause my music player without my hands? EXAMPLE"
Google: "We think you might like... Maybe this... Maybe this... Maybe this... Maybe this... Maybe this... "
Me: "Ah ha".

So now we are getting somewhere we just need to call our music player app with some extra bits “Music Player PAUSE”. Great. Hopefully it’ll work in our version of the software… So now to our other task, how to find when I’m not at my desk.

Third step

Google. Friend. Let’s talk.

Me: "Dear Google, how do I know when I am not at my desk?"
Google: "You location is here. Map. Have you seen our Google Doodle?"
Me: "Oh cool, a Google Doodle, lets see what it is. Oh, it's Larry Page's birthday, its a Google Doodle made out of other Google Doodles... so who's Larry Page..."
Google: "Larry Page's profile. You're telling me you don't know who Larry Page is? Sergey Brin's Profile. Next you'll be telling me you don't know who Sergey Brin is"
Me: "Who is Sergey Brin? Wait - wasn't I doing something?"
Google: "We always know what you are doing. Safe search on. You were looking for these ads: this one, this one, this one, this one, this one, this one, this one, this one.....
Me: "Concentrate"
Google: "A concentrate is a form of substance which has had the majority of its base component removed..."
Me: "Sigh. How does my computer know when I'm not at my desk?"
Google: "Maybe these pages will help..... (By the way, did we mention we have 2.5 gazillion results which took just 3 yoctoseconds to complete?)"

Distractions are everywhere – constant vigilance! (And a nice pair of headphones). Sometimes it’s difficult to even know what you need to “look for”. The best way to solve this one is guess, see what results you get, try some other words, and iterate until you get what you are looking for. In this particular case I should be asking “How does my operating system know when I’m idle”.

Me: "How does my operating system know when I'm idle EXAMPLE"
Google: "1.6 mil-blah, blah, try this page."
Me: "Ah, we just have to use that program, so lets read the documentation for that"

Now we know what app we need to use, we just need to attach this one to our previous one. This is where we actually start writing code…

Fourth step

So lets try our luck again. Hopefully we can simplify our code by finding someone else’s code which wraps around the apps we need. Yep, it’s another google. But first we have to decide on which programming language we will use – there are a lot of choices but it gets down to two choices:

  • What did I use before?
  • Is there something cooler?

If you choose the second one you’ll be working on this “simple” solution for the next month – this is supposed to be a quick thing so lets use what we are used too, (we might try the second one anyway, depends how bored, courageous, stupid we are feeling at the time). We’ll chose Python, we’ve used it before and there’s a large number of “Wrapped Apps” out there in the wild. So we add the word “Python” to our “music player” and “idle detection” searches.

Me: what?

Summary

  • There’s loads of things programmers don’t know how to do.
  • We are learning new things everyday.
  • We search the internet a lot.
  • It’s easy to get distracted.
  • Programming is way more complicated than it should be.
  • But it can be quite rewarding.
  • It can also be very time consuming.

Notes

I’ve tried to keep the jargon at a minimal. By “app” I mean any program, application, library or piece of code” – most projects use lots of combinations of these.

Posted on

WebViews – Seeing all your website.

So what’s a WebView? They are just small windows showing a webpage. Here’s an image of a page with four views on, click it to go to the page:

As you can see, WebViews provides a way to see many of your webpages at once, no need to load multiple pages or to click through, you just need to open this one page. Furthermore, it’ll do some error checking for you too.

Continue reading “WebViews – Seeing all your website.”

Posted on

SSH – A Brief Software Engineer’s Masterclass

SSH is a secure protocol for communicating between computers. There are many useful tools built on top of this protocol and they should be a part of every Software Engineers toolkit. This blog will detail how to connect to remote computers super quick and more securely, several ways to transfer files between computer (and edit them), and how to connect to ports so that services (such as databases) appear to be running locally.

Image of console using SSH

These commands are most easily ran on Linux OS’s but they have equivalents on other systems too. In particular the rise in popularity of mini computers, like the Raspberry Pi, which have no screens means these techniques are much more widely needed than ever.

Continue reading “SSH – A Brief Software Engineer’s Masterclass”

Posted on

Gaia star data with D3 – part 2 – prettier, faster

Welcome back! (if that statement isn’t appropriate you’ll want to check out part 1 first: Gaia star data with D3 – part 1). We are going to make everything look much better, and then do a couple of optimizations.

Let’s make the sky look more natural, i.e. Black. In the current SVG specification there’s no way to set a background colour, so to work around that we’ll place a rectangle across the whole image.

Continue reading “Gaia star data with D3 – part 2 – prettier, faster”

Posted on

Gaia star data with D3 – part 1

TL;DR: I’m going to show you how to take a fairly large dataset – in this case the galactic stars – and make some nice visualizations with a JavaScript library called D3. They look great.

Basic view of data.

Did you see the recent news from the ESA Gaia mission? It will eventually give the precise position and motion of one billion stars in our Galaxy (which is actually “only” one percent of the stellar population!). An initial release of data happened on 13th September 2016 and included the position, motion and distance of two million stars.

I wanted to take this data and play around with it and I like to play around with data with the help of D3. This is a web based visualisation tool, which unfortunately is restricted by the capabilities of today’s browsers. This means I’ve had to reduce the data points, however, I suspect we can still get some nice results.

Continue reading “Gaia star data with D3 – part 1”

Posted on

Generating Webpages

Check out my post about the work I’m doing at the University of Oxford’s eResearch Centre. It’s about taking semantically linked data and generating a useful, website like view of the data.

http://www.semanticaudio.ac.uk/blog/generating-webpages-for-digital-objects/

I’m using nodeJS and Dust with Jena’s Fuseki SPARQL database to select and display the data. More to come.

Full text follows.

Continue reading “Generating Webpages”

Posted on

Creating Excel files from CSV with Python

When you need to extract data from a system you can bet that someone will claim they have to have it in a Microsoft Excel format rather than the simple CSV format you had in mind. I’m unlikely to need to tell you this – as you’re already here – but that there are huge and annoying differences between these two formats. However, I’m not going to list them here, instead I want to introduce you to a nice python library: openpyxl ( https://pypi.python.org/pypi/openpyxl ).

This beautiful Excel library is a great example of how a good library should be written – it takes a complex problem and hides it away behind a nicely designed interface, and having written an Excel parser myself I can assure you that was unlikely to be an easy task! Basically, this library lets you read and write Excel files and requires next to no knowledge of the Excel format.

My little piece of code takes a list of CSV files with a name and inserts them all into a new excel file, you’ll get one sheet per CSV. It also improves the display by bolding the first line (assuming these are column titles) and adjusts the width of each column depending on the size of the strings within the column.

You can find the full code here: https://bitbucket.org/akademy/csvtoexcel/src/e92654d86ba35e221332c4365c78662b48f274c0/excel_writer.py?at=master&fileviewer=file-view-default

And you can use it like this:

	ew = ExcelWriter()
	ew.convert( {
		# Specify the csv files, whether it has column titles (default: yes) and the names
		"sheets" : [
			{
				"filelocation" : "test/work.csv",
				"sheetname" : "Works"
			},
			{
				"filelocation" : "test/person.csv",
				"sheetname" : "People",
				"has_titles" : False
			}
		],
		# Specify the output name
		"outputname" : "test/test_outout_file.xlsx"
	}

openpyxl can do much more – pretty much anything you can do in Excel – check out the docs for more info: https://openpyxl.readthedocs.io

So install the library and run the code!

Posted on

Node.js – the programmers “Save the World” tutorial

What this tutorial is

This tutorial was designed to help you understand how to create your own node.js program – not how to copy and paste someone else’s code! (However… you might have some luck finding the complete code at https://bitbucket.org/akademy/save-the-world/src).

It is assumed you are a programmer, with some experience of javascript and other programming languages such as Python, Ruby or Java. It’s really written for other programmers and tries to get to the points quickly (except for the odd alien invasion)!

It may be useful to imagine: Earth has only minutes to live and you are our only hope – your teacher is barking orders at you in a manic attempt to teach you the skills you’ll need to defeat the invading armada. You need to set up a node.js server before the laser bolts start burning! Quick!

Continue reading “Node.js – the programmers “Save the World” tutorial”

Posted on

KeePass with Putty sessions in ssh

This blog is about launching SSH sessions through putty but from within KeePass, mostly using Windows although other Operating Systems may also work. Putty essentially organises and launches SSH sessions between computers; KeePass is a Password Safe storage system. By manipulating the form of the URLs in KeePass we can simply right click an Entry and select open URL to launch a putty session.

KeePass and Putty

Continue reading “KeePass with Putty sessions in ssh”