Saturday, 29 April 2017

A quick thanks to Jason for the MIDI-in schematic (and opto-isolators)

If you're having trouble with your MIDI (as we were, trying to get our input working through an opto-isolator) there's only one person with the wealth of experience worth asking and that's Jason. He quickly identified that the random spare-part left-over opto-isolator we were trying to use for our MIDI-In simply wasn't up to the job.

Apparently there isn't a great range of opto-isolators to work from for handling MIDI. Most are simply used as switches. And while its' true that serial data is just flipping a switch really quickly, we need an opto-isolator that can switch on and off quickly enough to keep up with the data rate.

Jason kindly gave us a couple of 6N138 isolators and an updated schematic to work from.


The isolator neatly inverts the MIDI logic (in MIDI a zero is a high signal and a one is low) as well as holding the RX line high when idling - just how all good UART peripherals like it.

With this slight alteration we got our MIDI In working reliably - with the added bonus that we can now hook up and MIDI source, powered from anywhere; the isolator means there's no need for a common ground or similar reference. Which is just as well - to date we've tested our light-up guitar fretboard by loading MIDI files on the laptop and playing them through a usb-to-midi interface conntect to a second usb port. This means the serial port and the MIDI signal have both been generated from the same source.

But the ultimate test of our MIDI capabilities will be when we plug the guitar into a portable keyboard, press a key and see which of our frets lights up!

Friday, 28 April 2017

Displaying MIDI notes as fret positions

To convert scales and patterns into LED combinations we've made quite extensive use of spreadsheets. Boring old boxes of numbers have made displaying patterns of dots much easier.

The guitar fingerboard LEDs have been wired starting at fret 15, low string E, passing through each string of the 15th fret (E, A, G, D, B, E) then continuing with LED number six representing 14th fret low E, LED seven is the 14th fret A string and so on.

So we first drew up our patterns in a spreadsheet, coloured the boxes (to make them easier to identify) then transferred these patterns into byte arrays.


Anyone familiar with guitar scales might see the pentatonic minor box one starting with the A root note on the fifth fret in the image above.

After giving each note of the scale a "colour palette index", we stored each pair of strings in a single byte. So the low E string is the upper nibble of the first byte, the A string the lower nibble of the first byte, the D string is the upper nibble of the second byte and G is the lower nibble. The B string is the third byte upper nibble and lastly the high E string is represented by the lower nibble of the third byte.

So reading the scale chart from the bottom right corner, reading upwards, then from right to left, our fifteenth fret LED colour index values are 7(E), 2(A), 0(D), 0(G), 4(B), 7(e) (zero represents no dot to be displayed). The fourteenth colour index values are 0, 0, 5, 1, 0, 0.


You can see that by splitting each fret/string position into a single half-byte 0-F, when storing the values in hexadecimal, we can actually "see" the dots in the code. The first three byte values, for the six strings on the fifteenth fret are 0x72, 0x00, 0x47. The correlates directly with the dot patterns in the spreadsheet 7(E), 2(A), 0(D), 0(G), 4(B), 7(e)

Using a similar technique, we listed the note values of every fret position for every string.


Then after a quick Google look-up, recorded the MIDI note value for each note at each fret position, for each string. Recording the MIDI values in the same order we've wired the LEDs, our array of values begins 31, 36, 41, 46, 50, 55

These are the midi note values for G2, C3, F3, Bb3, D4, G4 which happen to be the notes on the guitar at the fifteenth fret.


Since the seven LED is at the low E string position on the 14th fret, our array of MID note positions continues 30, 35, 40, 45, 49, 54. We continue recording each note value in our MIDI array, all the way up from fret 15 to fret zero (open strings).

const byte midi_notes[] PROGMEM = {
  31,36,41,46,50,55,
  30,35,40,45,49,54,
  29,34,39,44,48,53,
  28,33,38,43,47,52,
  27,32,37,42,46,51,
  26,31,36,41,45,50,
  25,30,35,40,44,49,
  24,29,34,39,43,48,
  23,28,33,38,42,47,
  22,27,32,37,41,46,
  21,26,31,36,40,45,
  20,25,30,35,39,44,
  19,24,29,34,38,43,
  18,23,28,33,37,42,
  17,22,27,32,36,41,
  16,21,26,31,35,40
};

Now when we receive a MIDI note value, we can loop through this array to work out which LED corresponds to that note and light it up (or, if it's a note off message, turn it off).

void addNoteToFretboard(byte note, int col_index){
  // loop through all the midi note values in the array
  // and wherever there's a match, light up that LED with
  // the appropriate colour index
  byte p;
  CRGB c = RGBColour(col_index);

  for(int i=0; i<96; i++){
    p = pgm_read_byte(&midi_notes[i]);  
    if(p==note){
      led[i] = c;
    }
  }
  FastLED.show();
}

(in the example above, we have a function into which we pass an index number and it returns a CRGB value for a specific colour)

When hooked up to a MIDI source, the result looks something like this:



That's the opening sequence to Metallica's Enter Sandman. If you already know how to play the song, you'll recognise the familiar patterns around the 5th, 6th and 7th frets. For some notes, more than one dot appears. That's because there's more than one place where you can fret the note. Matching colours mean "it's the same note" so you'll see a red dot appear at both the 6th fret low E string and 1st fret A string at the same time. You can choose whether to play the riff on the 6th/7th fret, or on the 1st and second - it's up to you!

At the minute there's no differentiation between channels - you need to turn channels on and off using your MIDI sequencer - so when we wound the song on a bit and played it complete with distorted guitar, bass notes and drums/hi-hat, it looked a little bit crazy!




However, as a performance piece, we think it looks pretty cool too!

Thursday, 27 April 2017

Messing about with MIDI and RealTerm vs Putty

While our guitar neck glue is going off, and since we've pretty much got most of the LED goodness working (that'll probably have to be demonstrated in a later post, once everything has been stuck together) we thought the intervening time could be spent adding some extra cool functionality to our light-up guitar.

To date it allows you to select pentatonic or diatonic scales in any key, include extra notes (such as the "blue note" and flatted/major thirds in the minor pentatonic, major fourths in the major pentatonic and so on). You can also have it display all the major C.A.G.E.D chord shapes in any key too (we really should have made a demo video before this post!)

EDIT: made a quick demo video. Here you go -



In "offline mode" you can display different scales and CAGED chord shapes in any key. A simple flick switch will allow you to quickly change between major and minor (for those pesky I-to-IV and IV chord changes in a 12-bar song!) And, of course, there are some fancy patterns just for show as well!

Now most guitarists who know how to play guitar recognise that it's not just a Guitar Hero clone - where frets light up and if you match them with your fingers, you'll play a tune. By lighting up a particular scale (and different target notes from that scale) you can improvise blues-based solos more easily just by wandering between the dots. But for anyone wanting to play along to their favourite song, we figured it'd be a massive amount of work creating some software that allows you to input tab in such a format that you can then send to the guitar to get the frets to light up.

So when MIDI was suggested, we figured.... hmmmm.
It's always tricky knowing which of a possible five places you should place a dot for a specific note on a guitar. But if we could take the incoming MIDI signal and simply light up all possible alternatives, it would leave the player to decide which fret they found most comfortable to reach for (instead of some crazy algorithm going rogue and forcing you to whizz up and down the guitar neck at lightning speed!)

So we needed to get to grips with handling MIDI in signals (not the infinitely easier generating MIDI out messages). The first thing was to hook up a usb-to-MIDI device so we had an independent way of generating MIDI messages.
Now MIDI uses inverted logic (zero is represented by a HIGH signal, a one is a LOW signal) but other than that, it's simply serial data sent at a funny baud rate. 31250 bps to be precise.

So we downloaded a free MIDI sequencer (Anvil Studio looks gnarly and draws slowly, but it can create, edit and play MIDI files to a MIDI device, so it was good enough for this test) and grabbed something a bit more sophisticated than our usual serial favourite Putty - a little program called RealTerm.

Putty is fine for sniffing data, but RealTerm makes playing with serial data a doddle. Not only can it display the raw incoming data, but it has a myriad of options for decoding it too. We went with hex[space] so that we could visualise the MIDI values in hex, instead of trying to decode non-printable ASCII characters (as Putty likes to do). It's much easier to see what a value like 0xB4 means than a checkerboard patterned glyph!



We hooked up our MIDI out to a socket and introduced an opto-isolator to both isolate and invert the MIDI signal from our usb-to-MIDI device. The circuit was something similar to this one (with a few resistor values changed to match values we had lying around)


The strange this is, we got no data from the circuit. Absolutely nothing.
After fiddling about for about an hour and getting nowhere, we took a bit of a risk. Since the MIDI signal was being generated by a laptop usb port, we figured it wouldn't be more than 5v. So we connected the MIDI out directly to our serial in.....
Instead of a puff of blue smoke, we got serial data appearing in RealTerm. Success! (although we did have to invert the signal, with midi pin 4 connected to ground and midi pin 5 to our serial RX.


It might look like gibberish, but it's pure, raw, MIDI data!
The excellent MIDI resource site (https://www.midi.org/specifications/item/table-1-summary-of-midi-message) gave us an easy way of decoding the data - and thanks to RealTerm's hex view, we could even do it onscreen!


The MIDI messages we're interested in are the note on and note off ones. All other messages can be discarded. Most (but not necessarily all) MIDI messages arrive in three-byte packets. Luckily, the MIDI format also makes it really easy to decode.

The first byte of a MIDI message "packet" always has the first/MSB set to one. The first byte is also the "command" byte, telling us what action is to be carried out. Subsequent bytes, containing data values always have the first/MSB bit cleared (set to zero). So it's dead easy to find the start of each message - just read in bytes from the serial port and as soon as a byte has it's first bit set, you know you're at the first byte of a new packet.

We can see from our received MIDI data, we get a repeating "B" message, every three bytes at the very start. This is the MIDI device sending a "control change" message for each channel being used by our MIDI sequencer (since 0x0B in hex is 1011 in binary, which matches the pattern for "control change" in the MIDI specifications). The second character of the first byte is the channel number (from 0-15). The following two bytes are information about how the controller is set up (volume, left/right balance etc).

For the messages we're interested in,  we can just look for a message beginning with 9 (note on) or 8 (note off). If we dump our serial data capture into notepad and split the data into three-byte packets, it becomes much easier to read.


Our first musical MIDI instruction is 90 28 7F

This decodes as:

  • 9 = note on
  • 0 = channel zero
  • 0x28 = decimal 40 = octave1 note E
  • 0x7F = full volume


(note that normally we'd expect anything "full" to be 0xFF but in MIDI messages, the data bytes always have a leading bit zero, so the maximum value we can achieve is 0x7F)

And as anyone who has ever played "Enter Sandman" on a guitar will tell you, the first note struck is always the low E string. Unless you're Bill Bailey....



Things look quite promising so far - so what's next?
The next message is 90 34 7F. That's another full-volume, note on message, this time for note 0x34 (which is decimal 52, or octave 2 note E)

Following that comes 90 37 7F. Another full-volume note-on message, for note 0x37 (decimal 55, or octave 2 note G).

Now comes our first note off message - 80 28 50. It's that first note, low E. Which tells us that all three note have been ringing out until this time. Again, anyone familiar with the opening bars of Enter Sandman (that bit with the clean guitar tone at the start) will recognise that this is the case.


The reason the low E stops is because, when playing this on a guitar, you normally fret the low E string to play the two-note decending riff. Sure enough, the next messages representing the next two notes are

90 2E 7F
90 2D 7F

Even without decoding these to work out what the notes are, we can see that there's a half-step change in tone; whatever note 0x2E is, the next note is 0x2D, one half-step/semi-tone below it. Being familiar with the tune, we already know that this is correct!

What's interesting to note (although, to be honest, maybe not unless you're a bit of a music nerd) is that if these notes had been generated by a guitar, there'd be a note off between the decending tones. That's because both notes are played on the same string. There's no way that you could get both notes playing, unless they were played on two different strings (and just about everyone who's ever played Enter Sandman on a guitar normally plays it by moving down a fret on the same string). In our MIDI playback, somehow both notes are ringing out at the same time. It's only a few bytes further on that we see the note off message for 0x2E, followed a few bytes later by the note off message for 0x2D.

Our screenshot belies the fact that these on/off messages can appear one after the other, almost instantaneously. But it's just interesting to note that this tune was probably played out on a MIDI keyboard (where it's quite possible to sustain two notes next to each other) and the player's inability to get their fingers out of the way quickly enough has lead to some notes sustaining for slightly longer than they would if played on a different instrument.

Anyway, all that aside, we've now got a way of reading incoming MIDI messages and decoding them into note on and note off messages. In the next post, we'll look at how we can use this data to display the notes on our guitar neck....



Wednesday, 26 April 2017

Fixing the light-up guitar neck

One of the great things about having moved into the workshop bungalow is that I can try out ideas quickly and easily and designing through repetition is a doddle. In the past, I'd have to plan what I wanted to do/make then visit the unit, make it, bring it home and just hope it worked - at least until I could get to the unit a second time.

Now I can design something and whizz it out on the laser in minutes, not days. So when I got the RGB LEDs soldered up for Keith's guitar, I could play about with a few designs for sticking the fingerboard to the guitar neck.



If a design didn't quite work or fit properly, it took just moments to knock out another, amended version. It took maybe three or four goes to make some mdf standoffs to fit around the LEDs on the reverse side of the fingerboard


Because I'd positioned the LEDs by hand, rather than using a template or a PCB (where the position/location of the LEDs is fixed) they didn't exactly line up with the drawing I made the mdf template from. But it only took a bit of fiddling about to get a working fit.


Next we used some epoxy glue to fix one side of the mdf to the fingerboard, and clamped it down to a board to get the fingerboard as flat as possible.


Although not fully cured after a few hours, the glue had set enough to allow us to move the fingerboard and glue it to the guitar neck.


Now it's just a case of leaving overnight and see how things look in the morning! We used epoxy rather than PVA since many of the luthiers recommend it for it's stability. Apparently PVA can shrink over time, which might cause the fingerboard to pull or bow. Epoxy doesn't suffer from this (although "proper" luthiers recommend against epoxy as it makes a future repair almost impossible - if this thing goes wrong, the entire fingerboard will need to be sanded off!)

And there we have it - a light-up guitar neck, ready to be fit to the rest of the instrument. Because of the additional 2mm height added under the fingerboard, we're going to have to raise our bridge by the same amount (otherwise the strings will buzz as they "fret out" on the higher frets). A simple shim under the bridge raises it by 2mm, but this also means we're going to have to screw it down hard, losing the tremelo operation (having a moving trem might distort the shim under the bridge over time). Luckily the pickups can be adjusted by more than 2mm, so we've simply raised the nut, strings, bridge and pickups all by 2mm and the playability shouldn't be affected.


We'll have to assemble the rest of the guitar for testing.......

Sunday, 23 April 2017

Save some Arduino RAM when using strings with the F macro

Anyone who has ever written out debug messages to themselves while developing on Arduino will know, add too many and all your mcu RAM gets chewed up pretty quickly.

In "production" code, it's quite common to flash an LED to indicate what's going on, but that gets pretty tedious to debug when you're making lots of changes to your code as you develop, so it becomes common to little complex routines will little Serial.Println statements, to show where in the logic control you're up to.

A you might write something like

while(something){
  Serial.println("Here's what's going on");
}

after a few of these (ok, maybe like a couple of dozen or more) you'll find your RAM usage creeping up. Debugging code that uses wasteful libraries means either re-writing someone else's code (negating the benefits of a library-based development system) or reducing the message length (until you're doing little more than an alpha-numeric equivalent to flashing an LED).

More experienced users might write something like

while(something){
  Serial.println(F("Here's what's going on"));
}


At first the difference is difficult to spot. But that all important F macro (which isn't particularly well documented in Arduino help files) makes a massive difference. What that does is write your string message to program ROM rather than fill your RAM with pointers to character arrays that the Arduino string class uses.

Replace all instances of "my string" with F("my string") and you'll find your RAM usage plummets (while your program ROM size increases by roughly the same amount as you've saved with RAM).

We recently played about with the excellent nokia5110-compatible LCD library and built a rotary-encoder based menu system (for the light-up guitar I promised Keith). There are lots of strings of text used, and - sure enough - after coding a few menus, our RAM usage was on the up



While it seems trivial to add the F macro jut before each of our strings, in this particular case, it wouldn't actually work. See, our LcdString function accepts not an Arduino-type string object, but a pointer to a character array.

So if we tried to write LcdString(F(" string "));
it simply wouldn't work (the compile returns a data type mismatch error.

The answer is a quick-and-dirty function into which we can pass a string and return a character array, which we can then pass into our LcdString function.


Now we can write our string calls using the F-macro (to push the strings into program ROM space and free up RAM) but still pass them as character arrays into the functions that prefer character arrays over the Arduino string class.


In our menu test, we managed to conserve over 360 bytes just by implementing the F-macro using a string2char function. Given the atmega328 has 2kb of RAM but a massive 32kB of ROM, wherever possible we try to push our strings into ROM.

We managed to reduce our RAM usage by over a third (34%) for a modest 2% increase in program space. Given there's more to this project than just the menu system, we'd take any chance to reclaim back over 17% of the total available RAM for use in the rest of the program!

So next time you're getting close to using up all your RAM, find all those little debug messages and wrap them in the F-macro. And if you're passing strings into other functions, you can still use it and simply pass your F-strings into the string2char function if the function prefers a character array.


Saturday, 22 April 2017

Making a light-up-guitar for Keith

Having spent a few days in Dublin, I got chatting to my brother-in-law Keith. He was asking about the guitar project we worked on a while back and we swapped tales about learning (or failing to learn) the pentatonic scales and target notes properly.

When I got back I promised I'd build him a guitar to demonstrate how it all worked. Which was grand, except since moving into the workshop bungalow, I've not been able to find the massive PCBs to connect up the WS2812B RGB LEDs.

Not wanting to go back on promise meant only one thing - hand-solderingall 96 of the little buggers with a pair of tweezers and some thin-gauge wire. I thought I'd left wire-wrapping behind in the 90s! Luckily there were a couple of laser-cut fingerboards left over from building the last few guitars about a year ago, so I set about super-gluing some LEDs to the underside.


I got the idea from messing about with the electronic board game; having PCBs built for them would be prohibitively expensive, so we swapped the circuit boards for strips of copper tape and hand-building with loose components. By placing the LEDs the right way around, I figured I could connect the data_in and data_out pins together easily, then just join all the power and ground pins to two strips of copper tape on each fret.


It took nearly two days of positioning, soldering, testing, debugging, re-soldering - in between other work - but the end result was quite impressive


Connections between each strip of lights was made up the centre of the fingerboard so the outer sides could be glued to the guitar neck; since the truss rod is down the middle of the neck we wouldn't be gluing the centre of the fingerboard anyway.



A quick rainbow sketch on an Arduino with the FastLED library and we had a rather attractive display. Never mind lighting up frets, learning scales and showing how to play the guitar - I quite fancy another one of these with just the rainbow pattern. I might not be  the best performer at the next Open Mic Night at the Pebbles, but I'll certainly be the brightest!

Thursday, 20 April 2017

Unity, raycasting and line of sight between objects

We're putting together a simple 2D/top-down game that makes extensive use of "line-of-sight" rules as players move around the game world. There are a few ways you can check for line-of-sight but they almost always involve drawing a line between two points, then seeing which objects (if any) intersect this line.

If we were coding this in any other language for any other system, that's probably how we'd do it anyway; create an equation to describe the line between the two points, then "walk along" the length of the line, one pixel/unit at a time, checking to see if the x/y position of any other object in the gameworld is close enough to the line to be considered intersecting with it.

Unity provides this functionality already with its Physics.Raycast function.

Provide the function with a start point vector and a direction vector and it imagines an infinitely long line (the "ray") from the origin, and returns the first object that the ray collides with (if any). There's also Physics.RaycastAll which does the same thing, but returns an array of all objects hit by the ray.

We made use of the RaycastAll function in our line-of-sight checks (although we're building a 2D game, the same principles of 3D development still exist, we just treat everything as if it were all on the same Z-plane). We put three moving objects into our game world, with one of them hidden behind a wall. We then updated the position of each object and ran our line-of-sight checks from the moving object to all other objects in the world. The checks involved raycasting from the movign object to each other object (in turn) and checking the array of collisions.

If the array was empty, there were no detected obstacles along the line, and so we said that there was a line-of-sight between the two (in future development we'll have to include things like facing and field-of-vision and so on, but for now we're just trying to decide if an obstacle exists between two points). If any obstacle was returned in the array, we said that no line-of-sight existed between the two objects.



On the face of it, a simple Raycast function might do the job, as we're only interested - at this stage - in the binary option of "is there an obstacle between these two points". But we wanted to use the RaycastAll function to return ALL objects so that in future we might be able to assign "visibility" to different obstacles. Some obstacles may, for example, be see-through, but we still want them to act as an obstacle for purposes other than viewing. A classic example might be a glass window: you can see through it but it also acts as a physical barrier.

So we don't just want our line-of-sight function to return false if any old obstacle exists between two points - we want to inspect each obstacle type between the points and decide whether or not to include them in our line-of-sight check. So instead of Physics.Raycast, we used Physics.RaycastAll.

Everything seemed to be working just fine for a while; our hidden object remained hidden and the visible object revealed itself in good time. The function correctly identified whether or not there was a line-of-sight between all of the objects. Then something funny happened - despite there being a pefectly clear run between our first two objects, the LOS  function started returning false



Even more peculiarly, sometimes the function returned true (is there a line of sight between these two objects) and sometimes false, depending on which object we used as the source and which was the destination. Yet as we hadn't yet introduced rotation or facing into our function, it didn't make sense that an obstacle was found if we went from A to B but none were found if we went from B to A.

After much puzzling and re-reading the Unity documentation, we eventually worked out the problem. Our ray was continuing beyond the object being tested. So although we thought were asking "are there any obstacles along a ray between these two points?" the function was actually returning "are there any objects along an infinitely long ray, starting at point A and continuing in the direction towards point B?"



Of course, as soon as we moved an object so that there was a wall behind it, the function found the wall. The ray passed through the second object, struck the wall behind and said "yes, I found an obstacle along that ray".

What we needed to do was limit the length of the ray;
The RaycastAll function has an overload which allows you to enter a start point, a direction and a magnitude (maximum length of the ray). We created our ray be subtracting the gameworld co-ordinates of the source object from the co-ordinates of the destination object. This creates a vector describing the path between the two objects. We use this vector as our ray. Having created the ray, we then used the magnitude of the direction vector as the length of the ray.

As soon as we limited the length of the ray to match the length of the vector describing the direction from one object to the other,the function worked as expected, both "forwards" and "backwards" (i.e. it didn't match which object we used as the source and which was the destination).

bool hasLOS(GameObject source, GameObject dest){
   // firstly cast a ray between the two objects and see if there are any
   // obstacles inbetween (some obstacles have "partial visibility" in which
   // case we may or may not want to include as a "hit")

   RaycastHit[] hits;
   bool obj_hit = false;

   Vector3 dir = dest.transform.position - source.transform.position;
   Ray ry = new Ray ();
   ry.origin = source.transform.position;
   ry.direction = dir;

   hits = Physics.RaycastAll (ry, dir.magnitude);
   Debug.DrawRay (source.transform.position, dir, Color.cyan, 4.0f);

   foreach(RaycastHit hit in hits){
      // here we could look at an attached script (if one exists) on the object and
      // decide whether or not this should actually constitute a hit
      Debug.Log("LOS test hit from "+source.transform.position+" to "+dest.transform.position+" = "+hit.transform.parent.gameObject.name);
      obj_hit = true;
   }

   return(!obj_hit);
}


Within the foreach loop we can put some further testing to decide whether or not the obstacle has an effect. So in the case of firing a bullet at a target which is on the other side of a glass wall, we could call the function and ignore the glass object when testing for line of sight (can we see the object behind the glass) but include the object as an obstacle when using the same function to decide if, say, a bullet were to be fired from one object at another.

The same result could be achieved using trigonometry (lots of tan/cos functions) but Unity does provide lots of nice, easy, helper functions, such as Raycast and RaycastAll. Thanks Unity!