Two heads are better than one, an LEGO EV3 story.

An iPhone app to link it to your robot

In the summer of 2019 I did a gig to teach robotics using LEGO EV3 models. I planned to use ev3dev.org image and teach python which is exactly what I did.

But … I needed to make things fun for the younger students and wrote a remote control app to that end. An app that would work with the ev3dev.org OS, with the help of a python script on the robot side. I wrote about it here on medium on this link.

Moving ahead the summer course was a success, but left me thinking I could do more. What about instead of using my smartphone as a remote I built it into my robot itself, making my robot smarter.

This was the genesis of sensorCode, an app to run on a Apple iPhone to do just that, make your robot smarter. An app you can down load in this link.

Now your average iPhone has more than half a dozen sensors built into it. It knows where it is in the world, gps. It knows which direction it is pointing in, azimuth. It knows when it is moving and when it isn’t, motion. Some of the newer models even know how high they are in the world, altitude. Indeed it has some even more advanced sensors, namely a camera and a microphone.

My plan was to make as many of these available as I could. Cutting to the chase I built sensor code with six senses, one more than your average human :). Azimuth, motion, location, proximity, speaking and listening and also added the ability to switch on your iphone light using voice or commands too.

I used the the same protocol as I had with the remoteCode, namely TCP. But a colleague whom I managed to rope into the project Anton Vanhoucke of https://antonsmindstorms.com fame soon convinced me I needed to move to UDP.

Over the following months I built three proof of concepts, which Anton hallway tested for me helping me with the Python code into the bargain. Which brings me nicely to the first code byte in this article, a short python script of just 11 lines.

#!/usr/bin/python
import random
import socket
port = random.randint(1024,31766)
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(('', port))
s.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 1500)
print("listening on port ",port)
while True:
    compressed_data, server = s.recvfrom(1500)
    print(compressed_data)

Easy uh. You need just 11 lines to interface your robot with your smartphone. This script showing you the data your iphone is sending across.

But wait … stop I must confess I started running here before I could walk, or rather telling you how you how to run, before you could walk. To use this script, you of course need to download the app first.

You’ll find it on the apps store, under this link. Download it, install it and click thru the three intro texts. Select the “Connect” button and complete the two fields it asks for.

The IP address of your robot and the port number it should have shown you when you ran the script. Enter the details and you should get a message appearing on your robot’s console.

('listening on port ', 32766)
{"iphone":"192.168.1.123:49720"}

The first line is telling you which port your listening on, the second where this message is coming from.

Scroll thru the list of sensors, proximity, location and talking you get for nothing. The other four are one time in-app purchases, very inexpensive in-app purchases I dare say.

Try the location, it will report three very long numbers. Your latitude, longitude and altitude. Go to google maps, and copy and paste the first two figures separating them by a comma into the location field. The map you get back should be to your place!

It will report it along with all the other sensors you will be able to get data from. The default output is a JSON string.

{"word":"","id":"5e34595b","movement":{"roll":"","pitch":"","yaw":""},"proximity":"","position":{"altitude":"1240.9608154276879","longitude":"7.009915412385487","latitude":"46.33890448515321"},"direction":{"magneticNorth":"","trueNorth":""}}

You can parse directly in Python if you wish, you would use code that looks like this to do so. It will parse the location data you see above into a dictionary of values.

#!/usr/bin/python
import random
import socket
import json
port = 32000
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(('', port))
s.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 1500)
print("listening on port ",port)
while True:
 compressed_data, server = s.recvfrom(1500)
 print("raw ---> ",compressed_data)
 dictionary = json.loads(compressed_data)
 print("json --> ",dictionary['position']['latitude']," ",dictionary['position']['longitude'])

During trials some of colleagues I must confess were concerned that if you asked your phone to give you more data, like motion you would run out of CPU on your robot trying to parse it, so I added a few more options.

You can if you want thru the settings on the app and turn off the JSON, at which point you get something like this. Its a full record, but without any labels showing what’s what. Of course it is always the same length, you just need to work out which column is which.

,1580489456,,,,,1240.9608154296875,7.009915412385487,46.33890448515321,,

Be forewarned however that this works be striping out alphabetic characters, so might lose some functionality if you choose this option, voice for example that you can use to get your robot to react to your voice commands.

The middle path on this is to ask for a reduced JSON output, this would include only the sensors generating data. So in this case, this.

{"id":"5e345bfe","position":{"altitude":"1240.9608154296875","longitude":"7.009915412385487","latitude":"46.33890448515321"}}

Enough said, if you read this far… you sure should download the app now. This link.