12 Devs of Xmas
12 insightful articles introducing you to new and exciting thoughts and ideas in the world of web development
Sat, 04 Feb 2017 17:54:40 +0000
en-US
hourly
1
Day 12: Beat ‘dis: Re-addressing Audio Analysis
2017/01/day-12-beat-dis-re-addressing-audio-analysis/
2017/01/day-12-beat-dis-re-addressing-audio-analysis/#comments
Fri, 06 Jan 2017 00:00:56 +0000
?p=661
Happy first 2017 Friday! A long while ago, back in one of my actual real jobs, around 4pm every Friday I used to pick a random disco track and blast it into the office and dance around, making sure everyone was ready for the weekend. I think everyone really appreciated it… really
Here we are again – it’s Friday, the first full weekend of 2017 is nearly upon us, you made it through the first week! Let’s get some music going, add some disco visuals, party, and whilst we do all of this we get to do some maths too. It’s like a web devs dream over here 🤓
Really Ruth what is this all about?
I want to introduce a concept I’ve been considering for a while and something that needs me to start work on it. So this is the start, there’ll be more work to be done after this article, but it’s the new year, so let’s begin at the beginning.
If you don’t know me by now, I’m Ruth I would say ‘in my spare time’ here, but actually recently it’s ‘in my real time’ I create visualisations to music, in a browser, mostly with browser code (aka JavaScript). In 2016 I started a collective, it’s called { Live : JS }, it brings together a bunch of artists, like myself, who all build software and perform live shows with music, lights, projections, hardware, and, well JavaScript.
Yeh it’s crazy, it’s a little cool too, if you want to chat to us we have an open Slack Channel and we open source our code in our Github org too.
You were talking about audio?
Yes! Yes I was Specifically analysing audio, which is pretty important for what I do. Let’s take the analyser node of the web audio API for example – that’s pretty sweet. It returns velocity values for frequencies, (or volumes of individual pitch if that’s easier to understand). Up until now I’ve just been using those velocity values just as they are returned; let’s take this visual for example:
I’ve done it by creating a pretty small array to represent the frequencies, each ‘flare’ represents an item in the array, and as the velocity changes so does the size of that flare.
The thing is, it does that with frequencies between 0 Hertz and 20,000 Hertz. (Hertz is what we measure frequency in btw). OK that’s alright, that’s what the human ear hears.
That other thing is though – those super high frequencies don’t factor a lot in day to day sounds and not really in music, and I really want to start dealing specifically with music. So if we bear this in mind, we can reduce the range of frequencies we want to analyse.
And there is one more thing (probably the most significant). The web audio API means we receive back a linear distribution of those frequencies. Now that’s cool and everything but, let’s take a brief look at music frequencies… (have you read the word frequencies enough yet?).
Music notes are depicted by letters, namely A-G (there are sharps and flats, but for the purposes of this article we can concentrate on whole tones). The A note above the middle C note on a piano is 440 Hertz, the A an octave (8 notes/whole tones) below that is 220 Hertz, the A an octave above it is 880 Hertz. See what happened there, it halved and it doubled. That is a logarithmical scale, not a linear one.
So what’s really happening with this analysis is we’re getting data, but if we start being a little bit clever with that data, we can get better data.
I know right. Some days you gotta luv our jobs!
Why is this so important? Well let’s look at that demo again ⬆️ Yes it looks pretty cool, I made it myself, thank you ☺️ But I think it could look better. As the track plays, the vis itself stays pretty flat and there’s more activity around the bottom end of the spectrum (the top right in the case of the flares), and literally none at the top end (top left). This is all really due to our linear distribution and large range of frequencies detected.
Now I think I can pimp this visual, and not just this vis either. I’m currently knee deep in building out all my analysis, MIDI, visualisation hacks into a much more workable piece of software. A really great piece of functionality would be more control over the sound analysis.
Think about this: Beat detection – cool. High, mid and low frequencies – cool. Brightness – cool. We could even go as far as detecting what type of music is being played and adjusting things accordingly. Like if a piece of music is very bass-y then our high, mid and low frequency bands may differ to a piece of music that has overall higher sounds.
Now we can do this with the analysis node as is, but it would be better if the data we start with relates better to that which we need it to.
Oh I see your theory! How do we start?
Great, now we’re all on the same page. First thing’s first – reducing our frequency array. Instead of starting with a small array like before (we can end up choosing our array resolution later), we’ll chuck the item count up to get a more accurate analysis.
var freqDataArray = new Uint8Array(2048);
Now we could simply cut that array and just use the items from the first quarter, with the idea that up to 5,000 Hertz is the first 25% of an array representing 0 – 20,000 Hertz:
freqDataArray.slice(0,512);
Why 5,000 Hertz? This is where the majority of sound lies on the spectrum. We can adjust this value later depending on music we’re listening to and what instruments are used. I chose this value via an educated assumption, just as a starting point. (You can bear in mind for instance that speaking conversation sits under 3,000 Hertz).
Or we could filter the sound with the audio API considering we have that functionality anyway. Here I use the lowpass type of the biquadfilter which only let’s through frequencies in the sound up to 5,000 Hertz, at which I set.
const biQuadNodeFil = audioAPI.createBiquadFilter(),
biQuadNodeFil.type = "lowpass";
biQuadNodeFil.frequency.value = 5000;
audioTrack.connect(biQuadNodeFil);
biQuadNodeFil.connect(analyserNode);
analyserNode.connect(audioAPI.destination);
Check out the below CodePen, I’m spitting out the array of frequency velocities onto the page. When you hit the play button those values constantly start updating. If you toggle the filter on, you’ll notice the values in the top part of the array diminish. Also notice that the track playing hasn’t changed that much (as we’re still mostly within the range used in music).
If you’re interested there’s a cutOffVal var sitting in the JavaScript, you can reduce this down even more and see and hear more of a difference in frequencies being filtered out.
But it’s still a linear thingy bob…
Yes it is! We’ve only done the first part; capped the frequencies at 5,000 Hertz – yeah we could be missing a few high noises in certain musics, but for the purposes of this exercise this bit was the easy bit, so let’s keep it simple.
Caveat! ⚠️ I know your brain is flagging a little warning, the data is still going to be the data – regardless now of how we represent it in an array. Let’s take beat detection for example. One way of doing this is to look for a spike in the levels – which a certain number of array items go over a certain velocity (let’s say 75% of them go above the value 150), then there’s a spike in the music and that could be construed as a beat. We can detect that kind of thing on the array as it stands now or after we do the next stage of manipulation and we’d get very similar, if not the same, results. So if that’s all you were looking to do, you could probably just stop here and work with a smaller array and just be like , but if we wanted to look more into different ranges of frequencies (say I wanted my whole screen to ‘bulge’ on the bass) the next stage would be important. Read: There’s only so much we can do, this is essentially another stage of filtering, not re-analysing from the source.
So how do we spread apart these values and try to mimic the musical log scale. Here I’m going to pose one way and as I said at the top of the article this is the beginning of my studies in this area. I’m hoping as I progress with implementing better analysis within my applications, further ways will present themselves.
This method groups these array items, and averages out the values returned within them. The groups get bigger and bigger as the frequencies get higher, much like the gaps within frequencies get bigger the further up the piano we play.
function adjustFreqData() {
// get frequency data, remove obsolete
analyserNode.getByteFrequencyData(frequencyData);
frequencyData.slice(0,512);
var newFreqs = [], prevRangeStart = 0, prevItemCount = 0;
// looping for my new 16 items
for (let j=1; j<17; j++) {
// define sample size
var pow, itemCount, rangeStart;
if (j%2 === 1) {
pow = (j-1)/2;
} else {
pow = j/2;
}
itemCount = Math.pow(2, pow);
if (prevItemCount === 1) {
rangeStart = 0;
} else {
rangeStart = prevRangeStart + (prevItemCount/2);
}
// get average value, add to new array
var newValue = 0, total = 0;
for (let k=rangeStart; k<rangeStart+itemCount; k++) {
// add up items and divide by total
total += frequencyData[k];
newValue = total/itemCount;
}
newFreqs.push(newValue);
// update
prevItemCount = itemCount;
prevRangeStart = rangeStart;
}
return newFreqs;
}
Let’s take our modified array and use that in our original visualisation. Each part still represents an item in the array, it’s just you can visually see how better balanced the animation now is. Originally it was very bottom heavy, now the spread is more even and it moves more dramatically. Awesome.
Have a post NY party!
Chuck the music up, fullscreen your browser, celebrate 2017! It’s going to be a better year, surely ✨
So what’s next? Well back in November I started doing a bunch of experiments around audio vis in a browser for Codevember. This work is a little side step to add to the bigger picture, and make these vis more awesome. I’ll be sure to add some of this into the VJ software I’m starting to build too and any more work in this area I’m sure I will share.
Want to discuss anything in this article further? Just tweet me I’m always happy to chat about the things I write about, especially when they involve the audios
Credits: Big props to my dad for pairing with me on both the theory and code for this, there were times when I don’t think I understood myself but he still did. Also thanks to Kinnoha for happily providing some musical goodness for me to serve up, you can check out more of her ear magic on Soundcloud.
]]>
2017/01/day-12-beat-dis-re-addressing-audio-analysis/feed/
1
Day 11: Nudge Theory: an introduction
2017/01/day-11-nudge-theory-an-introduction/
2017/01/day-11-nudge-theory-an-introduction/#comments
Thu, 05 Jan 2017 00:00:49 +0000
?p=620
“You build it, you run it” is a fine principle, but it means you need to let your teams make their own choices. No one wants to support a system running on [insert inappropriate or flaky technology here] just because that’s the company’s recommended queue technology or data store.
But what’s the implication for ongoing support of your services when you end up with multiple CDNs, data stores, queuing technologies, issue-tracking systems, communication tools, build and deployment tools, and languages? It’s all fine when a big team is working on the new shiny thing, but what happens when they leave and you have five people supporting all the “legacy” stuff?
Relatedly, how can the technology leadership make sure program teams still pay attention to department goals that may not match their short-term incentives? You want to save costs on AWS; they want to get stuff out there and optimise VM size later.
At the Financial Times, teams are pretty empowered to make the right decisions for themselves, but this means they’re very resistant to top-down diktats. As a result, company leadership has to find other ways to influence people to do the right thing.
Luckily there’s a fair amount of information out there on how to influence people rather than force them to do things, particularly in the realm of government.
In this article, I’m going to describe what Nudge Theory is, and talk about why I think it is relevant for software development.
What is nudge theory?
“a ‘nudge’ is essentially a means of encouraging or guiding behaviour” – David Halpern, Inside the Nudge unit
Nudges try to influence you rather than force you: putting healthy food on display at eye level, rather than banning sales of junk food; or making sure there are litter bins available and signs explaining that most people throw litter in bins, rather than imposing fines on the spot.
Nudge theory was named and popularised in a book by Richard Thaler and Cass Sunstein. It’s proven attractive to governments, because it’s about small changes, avoiding legislation (which is costly) or financial incentives (again, costly).
Schiphol airport urinals
The canonical example is in Schiphol airport.
The black fleck in the picture is a painting of a fly. It turns out that men like to aim at something, and the fly is in the best place to avoid splashback. This simple change has reduced the cleaning costs by 20%.
UK Organ Donation Register
Another example relates to the UK organ donation register. In the UK, this is an opt in register, and although 90% of people support the idea of organ donation, only 30% of people have signed up to the register.
When you renew your car tax online, you’re prompted to sign up. The Nudge Unit ran a randomised controlled trial, with eight alternative versions of the sign-up screen.
Those variants were:
1 – the control
2 – states what other people do in the same situation. There’s evidence that we’re affected by social norms
3 and 4 do the same thing as 2, but with pictures added. Previous research suggested adding relevant pictures increases the chance of someone donating to charity
5 – framed in terms of negative consequences
6 – framed in terms of positive consequences
7 – framed in terms of reciprocity – do to others what you want done to you
8 – pointing out the gap between the 90% that support organ donations vs the 30% that actually register
The results were statistically significant:
Reciprocity does the best. Notable is that adding a picture of ‘people’ to the social norms option was WORSE than the control, and framing in terms of negative outcome worked better than framing in terms of positive outcome.
What does this mean in terms of numbers? Well, in a year the difference between control and the best alternative would be around 96,000 extra registrations. Given there are about 21.8 million people registered in total, that’s about a 0.4% increase in the total number of organ donors for the sake of a few changes to a web page.
How can we apply this to software development?
The Nudge Unit came up with an acronym, EAST. They suggest that effective nudges are:
Easy
Attractive
Social
Timely
I think we can use each of these aspects to more effectively influence teams to do the things we care about as a company.
Easy
If you want someone to do something, you should make it easy: both to understand what the benefit is and to take the appropriate action.
Clear, simple messages, and a good choice of defaults go a long way. We have a strong tendency to stick with the default, which is why it matters whether something is opt in or opt out.
Things we can do to make stuff easy:
Supply checklists, APIs, example code, libraries.
Allow people to try your stuff out without having to wait for someone to allocate a key or set up a user account.
Be customer-focused: make sure people know who to contact. Anytime I see a tumbleweed icon in a team’s slack channel, I wince.
At the FT, we have APIs for creating change requests, which have replaced Salesforce forms.
However, the team have made things even easier by also supplying shell scripts and github webhooks. I integrated the change requests for our systems in minutes thanks to this.
We use the power of defaults with our AWS instances, which default in our staging environments to being shut down overnight and at weekends. Teams have to actively choose to keep them running outside normal working hours rather than actively choosing to turn them off.
Attractive
There are two senses in which you need to make something attractive: firstly, you need to attract people’s attention so they know what you are asking them to do. Then secondly, you need to make that thing attractive by explaining why they should want to do this.
An example at the FT comes from our security team. They want us to use WhiteSource, a tool for scanning libraries in various languages to look for known vulnerabilities.
They’ve documented it comprehensively to explain what it is and why we should use it. They also have a one pager for getting started that tells you exactly what you need to do, and also shows the languages that can use this: I’d assumed there wouldn’t yet be support for Go, but I quickly realised from this one pager that my assumption was wrong.
Social
Humans are social animals. We’re influenced by what other people do: sending someone a letter telling them 95% of people pay their tax on time is proven to make it more likely that they do that too.
In software development, we can show how other people are doing: for example, wherever possible we want our teams at the FT to migrate to Amazon Linux, because it saves us money – and we have a website that shows each team how much they would save:
We also encourage people to talk about things that have worked for them, in lightning talks or blog posts. When we share information we can show how easy it is to do something and show what people can gain from it. My team adopted Graphite and Grafana because of a really good lightning talk that showed how useful it would be and how easy it would be to integrate it with our systems.
Timely
It’s important to pick the right time to ask people to do things. Usually, that’s when they are just starting to think about how they are going to approach their problem.
If you can provide a solution that already provides most of what the team wants, people are very likely to opt for that.
We have an Engineering checklist at the FT that covers the things we expect teams to do.
Here’s part of it:
For each item on the checklist, we link to documentation, libraries, code examples etc. When teams need to get something done, they’re given all the relevant information right then and there.
Having a checklist also makes it easy to do the right thing, and applies some social pressure on teams to do what the other teams are also doing, so it works across several of the levers.
Conclusion
My colleague Matt Chadburn has a great blog post about how a free market economy can be a great thing within a company: let teams pick the best tools, either internal or external. This encourages internal teams to behave like service providers: they need to build great tools, easy to use, well documented, fit for purpose.
Internal teams don’t have a captive market: but they should have a massive advantage. They have their customers right there, and they only need to build tools for a single specific situation.
I think our tooling teams now understand this. They changed the way they worked. They started creating small individual tools and APIs. This means as a client, I can pick and choose. It’s a bit like the unix philosophy in that it favours composability over monolithic design. We now have a set of tools that each do one thing, and do it well. The teams can then compose those however they wish.
But how can an internal team persuade other teams to pick their tools rather than going externally? By making sure those teams know how easy it is to use the tools, the benefits they can get from them, that other people are using them successfully, and making sure they get this information at a time that’s relevant to them. This means nudge theory and EAST can help.
]]>
2017/01/day-11-nudge-theory-an-introduction/feed/
1
Day 10: The WordPress REST API: A Christmas Present for Front-End Developers
2017/01/day-10-the-wordpress-rest-api-a-christmas-present-for-front-end-developers/
2017/01/day-10-the-wordpress-rest-api-a-christmas-present-for-front-end-developers/#respond
Wed, 04 Jan 2017 00:00:21 +0000
?p=643