12 June 2016

Eyeball - A Minimal Neurorobot

This is my second post on personal neurorobotics. In the previous post I outlined the case for brain-based robots as consumer products, especially in education. In this post I will describe a minimal personal neurorobot that I call Eyeball. Eyeball implements a causal loop that is fundamental to animal (and neurorobot) behavior: signals travel from the brain to motors, causing behavior and change in the outside world, which is perceived by the brain as visual or other sensory feedback.

Eyeball is a web-camera attached to a servo motor. Eyeball's brain consists of two spontaneously spiking neurons that run on a USB-connected computer (eyeball.py, requires OpenCV, windows installation instructions here). One of the neurons is a motor neuron: whenever it spikes the motor moves to a new position. This changes the field of view of the web-camera, which continuously sends video frames back to the brain. The second neuron is a sensory neuron that is maximally activated by a dark spot on a white background. The spikes of the sensory neuron inhibit the motor neuron. This means that when Eyeball encounters a dark spot on a white background it stops moving and fixates on it.

Eyeball is very cheap. Web-cameras and servos can be bought for less than $5 each. To send serial commands via USB from Python to Eyeball I use an FTDI chip, which can also be purchased for around $5. I used an Arduino board to convert the serial commands to the PWM format needed to control the servo but only because I don't yet know how to send PWM commands from Python directly. So in principle the device costs less than $20. (Of course, to be successful Eyeball would also need a nice-looking plastic case.)

Despite the low cost, Eyeball has all the components needed to emulate some interesting brain functions and behavior. Vision is perhaps the best understood of all brain functions, and brain-based models of visual object recognition such as HMAX are already used to give neurorobots vision, but only in academia - not yet for consumer-oriented educational applications. Given the ability to recognize objects, Eyeball could moreover be trained to orient towards and track some objects and avoid others, ideally using a realistic implementation of the tectum/superior colliculus and basal ganglia. A reward-button could be used to deliver a dopamine-reward, changing synaptic weights according to known learning rules and thus training the robot to show preference for some objects. Easy access to the web-camera's microphone and the computer's speakers opens the door for voice communication... etc... etc...

While even this very simple neurorobot opens up a lot of interesting possibilities for implementing and exploring mechanistic models of the brain, what we really want is a robot that has two eyes and can move around independently. This will be the topic of the next post.

26 April 2016

Personal Neurorobotics

I've been exploring this idea for a few years now. It's time I start to document what I'm doing and ask for feedback. This is a summary, I'll go into detail in future posts.

Consider these trends:
  • Mechanistic models of brains and brain functions are getting better and better
  • Smartphones and laptops are becoming powerful enough to run such models
  • Hardware is getting cheap enough to build robots that can see, hear, make sounds, move around and communicate wirelessly for less than $200
I believe these trends open up a market for autonomous robots whose control-systems emulate biological brains. Consumer robotics is already a rapidly growing phenomenon and neurorobotics is an expanding area of research but I have yet to see the two endeavors combined. Please let me know in the comments below if there's some product or project I've missed.

A first generation personal neurorobot might emulate the brain of a fish or lamprey. Done right, it would explore its environment, avoid obstacles, escape threats, pursue desired states and objects, and learn, both from experience and explicit training. While these are behaviors that conventional robots can be programmed to perform, the defining feature of a neurorobot is its realistic brain architecture and activity, which makes it ideal for exploring and teaching neuroscience. Here, all the robot's brain processes could run on a wirelessly connected smartphone or computer, and be available for observation, explanation and modification in real-time. Using reinforcement learning to train the robot would be particularly interesting.

I see two markets for personal neurorobots:
  • Schools. Robots are already used in schools (e.g. FIRST Robotics Competition, LEGO Education). Neurorobots add the possibility of teaching neuroscience and behavior, which broadens the appeal substantially.
  • Enthusiasts. The popularity of neuroscience on the one hand and of consumer robotics on the other indicates that there would be a lot of interest in a project that combines both.
Importantly, the complexity of the brains and behavior of personal neurorobots could be increased year by year as new neuroscientific findings and models become available. Hopefully this can be an open process, with both individual enthusiasts and larger research teams working to make new brain circuits and capabilities available to the broader user-base. (The Human Brain Project is betting on a similar dynamic with their simulated robot testing environment, the Neurorobotics Platform.) The long-term aim would be to emulate the brain functions of higher vertebrates, such as complex learning, communication, attachment, planning, language and play.

While I think this project needs the attention of experienced roboticists, programmers, educators and investors, I will do my best to demonstrate feasibility and promote the idea. In my spare time I've developed a prototype neurorobot that I call a vertebot, pictured above. The hardware works, although ideally a personal neurorobot should be a light off-the-shelf product, not a five-pound beast that's prone to burst into flames. My main challenge is the code. I'm only fluent in Matlab and this project requires a non-proprietary language. So I'm learning Python and I'll ask for help with that as I move forward. Currently I can just about make a single neuron spike (single_neuron.py). I've set up a website (www.vertebot.com) where I'll put code and up-to-date information on the project. I'm also setting up a GitHub repo. All other observations and ideas I'll share here on the blog.

It's a big, sprawling project. I hope you'll find it interesting.

24 April 2016

Best Podcasts 2016

It's a great year in politics so let's start there. Slate's Political Gabfest is still going strong. Co-host John Dickerson has moved up in the world of US politics, he now moderates presidential debates and interviews the candidates on CBS's Face the Nation (don't miss his FTN Diary). However, the Gabfest faces formidable election coverage competition from the Five Thirty Eight elections podcast, which has a great, geeky ambiance and a fresh, numbers-based approach to punditry. Vox's The Weeds, The Econonist's various podcasts, and shows like The Glenn Show on Bloggingheads TV do good analysis of news and society. BBC's Newshour is still the go-to podcast for big complex stories like the Panama Papers.

Long-form interviews can be extraordinarily good when done right. WTF with Marc Maron, CNN's The Axe Files with David Axelrod, and The Ezra Klein Show frequently hit the mark.

Very Bad Wizards is the best podcast. Period. Social psychology, moral philosophy and great banter. I own the T-shirt. Repugnant. Space Time Mind, another philosophy podcast was really picking up steam (the Transhumanist Hot Tub episode is absolutely genius) but has been silent for months.

When it comes neuroscience, Neuroscientists Talk Shop wipes the floor with the competition. Try it (the episodes with Kreizer and Canavier are two recent highlights) and then try something like Nature's Neuropod or the Brain Science podcast and decide for yourself. (That said, yesterday I discovered Brain Matters, which looks like it could be really good. Fingers crossed.)

At the end if the week, relax with BBC's Friday Night Comedy podcast The News Quiz. Sandi Toksvig left to host QI and has been gloriously replaced by Miles Jupp.

Finally technology. Leo Laporte is still king. This Week in Google is pretty much always very good. Security Now, This Week in Law and This Week in Tech can be good if the topic and panel are. Non-TWiT podcasts worth keeping an eye on include Robohub's Robots podcast, the Gillmor Gang, and of course Seminars About Long-Term Thinking.

And don't forget In Our Time.

I use the paid-for version of Podcast Addict on Android.

Please let me know about any podcasts you think I would like.


Wernicke's Area, where podcast do their magic.
Polygon data were generated by Database Center for Life Science(DBCLS)[2]. Polygon data are from BodyParts3D[1], CC BY-SA 2.1 jp, https://commons.wikimedia.org/w/index.php?curid=32534031

24 March 2013

Musing on the mind-brain problem

Following a couple of recent conversations with friends and family I've written a short summary of my current views on the mind-brain problem. The dry jargon of scientific research reports is an obvious obstacle to a generally satisfying account of the conscious self, so I try to not be dry. This is just my current perspective, I'm not providing references and I reserve the right to be wrong.

I think it's important not to think of the brain as 'just a bunch of cells', but rather as a hundred billion individual identities that want to live and grow. The ancestors of the cells of the brain were free agents; swimming, creeping, crawling, swirling their way through the waters of ancient earth; feeding, resting, sensing, fighting, fleeing and multiplying. Now they're here, living together in this civilization we call brain; but they are still feral. In a very real way they rival and tussle every day to stay alive. The neurons of the brain do not grow old and die like other cells; you have almost entirely the same brain cells now as you had when you were a child. However, tens of thousands of them are wiped out every day - only the ones that form important constellations and alliances with other neurons receive 'neuromodulators' and grow; others shrivel and fade. Neuromodulators, what Gerald Edelman called 'value systems', are essential to the life and growth that brain cells seek. And here is the essential fact: neuromodulators are released in the brain in response to meaningful events of various kinds, happenings internal or external that bear on the interests of the body, the person, the brain as a whole, or one of its neural communities. The brain cells, seeking neuromodulators, seeking life and growth, are therefore in constant electrical communication and structural flux, seeking to bring about, probe and explore the meaningful, important aspects of the reality in which they find themselves.

What are these aspects of interest, of meaning, that allow neurons to survive and grow? What happenings attract the complexity and potential of a living human brain cell? To start with, every neuron is in constant electrical union with the sensing, moving body, and the neurons communicate and grow about this vital fact. The neurons share a common path through life, and so they explore and probe their shared memories constantly for nuggets of intrigue. Although their communities are often in tension as each continues the ancient will to live on and grow against the daily weeding out of the least relevant of them, they nevertheless share a common mouth, a common pair of hands and eyes, and the electrical urges of hunger, need, sleep and dreams reverberate across the neural fields endlessly. How could they not share a sense of I in this circumstance? From this seeking, seething, astronomically complex swirl of electric neural energy, membrane and will to survive and grow emerges pleasure and frustration, I and not-I, hopes, plans, dreams and distinctions. The astounding communities and constellations of living, electrified tissue that constitute each of these core features of the human experience are there for us to explore, by any method we choose - introspective, statistical, fictional, spiritual, communal - and it is our tremendous fortune and grace to be alive just as the technology to express and understand all this is finally beginning to become available.

At the heart of it all then, is a dynamic, inventive, persevering civilization of cells, seeking nourishment, excitement, love and force, in a never-ending myriad of ways. This is what it is to be alive, a conscious human being; a near-instantaneous sharing of memory, will and rich experience among the one hundred billion little lives within that one skull. It is in their nature to seek, like their cousins still independent in the sea; but the cells of the brain seek in communication and structural union with other brain cells. The subject of this electrical conversation is and feels like you.

25 December 2012

Coupled neural attractors

Towards the end of my PhD I made a second audiovisual rendering of feeding circuit data (the first is quite different and is available here)

For those of you looking to do something similar, this blog post explains how it's done (mainly Matlab). Code and data are available here: neuralAttractor.m, nineSpikeRates.txt. Suggested improvements are welcome too.


The input data for the video is 1000 seconds of spike time data from nine neurons that has been converted into smooth normalized spike rates. The data was recorded from two bilateral triplets of motorneurons and one triplet of 'extra-CPG' neurons that adapt the feeding behavior of the six motorneurons to the availability of food (see Harris et al. 2012 for details).
data = load('nineSpikeRates.txt');


The video is made entirely in Matlab. Use this line to initialize and set the length of the movie
videoFrames = moviein(length(data)-10); 
Then, for t = 1:length(data)-10, run
dataSegment = data(t:t+9, :);
plot3(dataSegment(:,1), dataSegment(:,2), dataSegment(:,3), 'o', 'Color', 'r', 'MarkerSize', 30);
hold on
plot3(dataSegment(:,4), dataSegment(:,5), dataSegment(:,6), 'o', 'Color', 'r', 'MarkerSize', 30);
plot3(dataSegment(:,7), dataSegment(:,8), dataSegment(:,9), 'o', 'Color', 'b', 'MarkerSize', 30);
axis([0 1 0 1 0 1])
grid on
videoFrames(t) = getframe(1);
This creates and saves multiple 3D plots showing successive 10 second segments of neuron triplet activity.

Finally, to save the video, use this line
movie2avi(videoFrames, 'videoOut.avi', 'compression', 'none', 'FPS', 10);


To play music in Matlab I use the Java library JFugue. Download the library jfugue-4.0.3.jar, or whatever is the latest version, from jfugue.org. Use the following lines to make the library available to Matlab. Note that the javaaddpath command doesn't like to be executed as part of a script - you may get an IMPORT error. To get around this you can run the javaaddpath line manually before you run the script (this is possibly a newbie error on my part).
import org.jfugue.Player
import java.io.File
player = Player()
The JFugue player takes a single long string as input. Each note is represented by an integer between 0 and 127 enclosed in square brackets (JFugue in fact supports a very wide range of musical commands, like 'Cmaj'). For example, try:
notes = '[48] [50] [52] [53] [55] [57] [59] Cmaj'
JFugue can play several notes simultaneously. Each channel or 'voice' is indicated by V1, V2 etc. You can specify an instrument for each channel, e.g. I0 for piano, I24 for guitar etc. You'll also want to specify a tempo for the entire string, using T[beats per minute]. For example:

notes = 'T[120] V1 I0 [48] [50] [52] [53] [55] [57] [59] [60] V2 I24 [60] [59] [57] [55] [53] [52] [50] [48]'

To generate the audio for this video I initialized the JFugue string notes = 'T[600] ' and ran the following for n = 1:3
notes = [notes horzcat('V', num2str(n), ' I11 ')];
for t = 10:length(data)
   note = round(mean(data(t,[1 2 3]*n)*30+30));
   if note > 30
      notes = [notes horzcat('[', num2str(note), '] ')];
      notes = [notes 'R '];
This takes the mean spike rate of each neuron triplet at each time step (t) and converts it to a bracketed integer between 30 and 60 (a mid-pitch range) unless the spike rate is zero, in which case 'R ' is added, which JFugue interprets as a pause. t starts from 10 here so that the sound played at each time step will correspond to the front of the 10 second segments of activity used to generate the video frames.

Use this to save your audio as a midi file
filePath = File('audioOut.mid')
player.saveMidi(notes, filePath)


Now you need to merge video and audio. As far as I know there's no way to do this in Matlab. JFugue only outputs midi files and unfortunately Windows Movie Maker doesn't accept midi, so if that's what you're planning to use to merge you'll need to convert the midi to wav first. I use Switch Sound File Converter for this. You may need to play around with the number of beats per minute at the head of the JFugue string to get audio that's the same duration as the video, especially if you're using many data points per second. If you play the YouTube movie above to the end you'll see that I didn't get this completely right.

An example of the video file I get after merging videoOut.avi with audioOut.wav (this is audioOut.mid converted to wav) in Windows Movie Maker is available here. Good luck.

23 August 2012

What is reward?

So I recently passed my PhD viva and got a paper published (whoop whoop!). The titles of the two texts are ‘Multi-electrode analysis of pattern generation and its adaptation to reward and ‘Multi-neuronal refractory period adapts centrally generated behaviour to reward. That last word, and my use of it in the texts, caused a fair amount of trouble. I na├»vely thought it would be OK to leave the term undefined, seeing as we’re still working out how the brain’s reward system operates. For example, the observation that midbrain dopamine neurons are sometimes activated by stimuli purely by virtue of those stimuli being new and unexpected (rather than appetizing, sexual etc) suggests that novelty itself might be thought of as a reward. And anyway we all have a fairly good intuitive understanding of what constitutes a reward, right? Wrong. I need to define reward.

Here’s what we ended up writing in the paper following extended skirmish with reviewers:
"We will refer to… a ‘reward’ in the general meaning of a stimulus that promotes approach and consummatory behaviour rather than the more specific meaning of an unconditioned stimulus used as a positive reinforcer in a classical or operant long-term conditioning paradigm." (Harris et al., 2012)
I'd like to contrast this definition with Wolfram Schultz's, who writes:
"A reward is any object or event that generates approach behavior and consumption, produces learning of such behavior, and is an outcome of decision making." (Schultz, 2007)
Schultz's second condition, that rewards produce learning of approach behaviour and consumption, begs the question: does this refer to conditioning proper, in which memory persists long after the reward is removed, or does an effect on short-term memory suffice? For example, is a food object rewarding merely by virtue of inducing a high and sustained feeding rate, or must it also increase the probability that similar food objects be eaten in the future? This question has physiological consequences: both classical and operant conditioning require brief bursts of spikes in midbrain dopamine neurons (Tsai et al., 2009; Kim et al., 2012), whereas the rate and intensity of ongoing behaviour, and the stability of working memory representations, are regulated by the tonic concentration of dopamine, which is set by the number of dopamine neurons engaged in slow pacemaker firing at any given moment (Niv et al., 2007; Cools & Robbins, 2004). In fact, Schultz's definition of reward does require persistent memory formation, i.e. bursts of dopamine. I disagree. I think a stimulus-induced increase in the rate and intensity of approach and consummatory behaviour can be thought of as a reward-response regardless of whether it produces lasting behavioural change. Yael Niv has for example argued convincingly that the average rate of reward over time modulates tonic background concentrations of dopamine, and thereby adapts the rate and intensity of foraging behaviour (Niv et al., 2007). There are many indications that this extends also to non-food rewards. This view is also in accordance with Norman White's, who writes that rewards are stimuli that elicit approach behaviour whereas reinforcers induce memory consolidation (White, 1989). Roy Wise similarly notes that 'priming' is an important effect of rewards, but one which does not find its way into long-term memory (Wise, 2009).

Schultz's third condition, that rewards be the outcome of decision making, is also problematic. If this condition is taken to mean that a reward must be the consequence of an overt motor behaviour, as many people would argue, then two objections follow. First, cases of classical conditioning where a neutral stimulus is paired with for example food, producing a subsequent preference for the neutral stimulus, do not involve any overt motor behaviour or action and so cannot according to this definition be said to involve reward. This is in stark contrast to numerous papers that describe such experiments as 'classical reward conditioning' and the food stimuli used as rewards. Second, say you give a hungry rat a food pellet, either at a randomly chosen time or as a consequence of the rat wandering into a pre-defined part of the cage. Do we really want to say that the pellet is a reward in the latter case but not in the former? Physiologically there will be no difference: the dopamine burst response and its effect on synaptic plasticity will be the same. Isn't it in fact the case that brains are always in the process of  deciding how to act, and operate by responding to correlations between their own activity states (be they sensory- or motor-states) and varying concentrations of dopamine? Whether or not a reward is in fact the causal outcome of a decision is irrelevant from the perspective of the brain.

In light of all this, I would suggest the following new definition of reward:
A reward is an object or event that induces approach and consummatory behaviour, and produces short- or long-term learning of that behaviour.
The lack of reference to rewards necessarily being the outcome of overt decision making constitutes a deviation from the way the term reward is used in everyday language (for example, an unexpected tax-return is a reward according to this new definition), but not, I think, from the way many scientists use the term. One might argue that such stimuli should be referred to as 'non-contingent rewards', but, at least in the case of the term 'reinforcement', this approach appears only to have complicated matters (Poling & Normand, 1999). Maybe then, we should drop the term reward entirely, and use 'positive stimulus' instead? However, this term has the serious disadvantage of not being a verbal noun. That is, whereas everyone understands the noun 'reward' and the associated verb 'rewarding', there is no established understanding of the (compound) verb 'positively stimulating' that is associated with the (compound) noun 'positive stimulus'. If anything, 'positive' has optimistic or ethical connotations that would jar with the amoral and downright destructive topics often discussed in relation to reward, such as addiction. The term 'appetitive stimulus' (and 'appetitively stimulating') avoids this problem but implies a focus on satisfying bodily needs, particularly hunger, whereas the key property of reward is that it can apply to any desire or goal.

Have I missed something; some word with the same meaning as 'reward' but better able to match the physiology? Is it time to make up a new word? If not, then I would suggest we stick with reward, using the definition above, accepting it as a slight neologism. The lack of a requirement that a reward necessarily be a consequence of overt decision making or motor behaviour should be appropriately tempered by the understanding that in fact the vast majority of rewards do occur as a consequence of decision making and motor behaviour - specifically as the result of exploration, trial-and-error, or more complex goal-oriented behaviours.

And here we see the fuck yeah monkey upon his mountain of treats!
(The treats are all rewards provided the monkey has an appetite)
(h/t Austen)