These are the voyages of Diesel, the Russian Dwarf hamster...
Absolutely no hamsters were harmed in the making of this project, as it is entirely done via remote sensing! As far as we can tell,
Diesel's privacy has not been invaded; he still pees in a secret corner and buries other waste products - and he has so far to lodge a complaint
to the house council despite being given ample notice and chances to do so. A plus side of monitoring him is that, perhaps in theory, his activity level
may even be a proxy for his health and happiness - hamsters do love a good spin on their wheels!
Identified "need" - how do we know how healthy our hamster is? When does he do his exercise? How fast does he run? How far does he run?
Let's get a reed switch and a magnet on his wheel, and do some calculations using C++ on an Espressif ESP32 WROOM32 DEVKIT1 that I've got lying around...
(I'd already proved the code concept using a Teensy 2.0 (Arduino-like chip), counting wheel revolutions and basic maths to calculate activity time/etc.)
This code step was simply to test the WiFi capabilities of the ESP32 and run a rudimentary webserver displaying the status of a connected switch.
16/12/24 - Adding wheel spin data
Iterative update to the code to calculate wheel circumference (i.e., diameter * pi) and a cumulative log of distance (i.e.,
distance = distance + circumference every time the wheel spun round by one revolution. Capturing a wheel spin event
is not as simple as if (digitalRead(reedSwitch)) {wheelSpins ++;} because this would generate a rapid increase in the value
of wheelSpins for however long the switch was active. Instead first a varaible reedSwitchState must be set to high on detecting a
trigger, and and if() statement created to only increment wheelSpins if the value was previously == 0. The resetting of the switch to LOW did the opposite,
i.e., setting reedSwitchState to LOW as well, and only doing so if the value was previously == 1:
if (digitalRead(pinSwitch)) {
if (switchPressed == 0) {
digitalWrite(led, 1);
digitalWrite(ledReed, 1);
delay(10);
digitalWrite(led, 0);
digitalWrite(ledReed, 0); // at this time an LED was connected as a debugging tool to determine the if statements were working.
switchPressed = 1;
wheelLast = millis();
Serial.print("Wheel spin at ");
Serial.println(millis()/1000);
switchCount++;
}
} else {
if (switchPressed == 1) {
switchPressed = 0;
}
} //end if pinSwitch
Instantaneous speed was calculated
by taking the millis() value at the time the reed switch was triggered last time and subtracting this from the current millis()
value, then dividing the wheel circumference by this difference in time. Top speed was simply determined by
if (speednow > maxspeed) {maxspeed = speednow};The ESP32's webpage at this point simply dumped the variables
reed switch state and distance travelled to its only response page.
16/12/24 - Adding wheelspin data
Iterative update to the code to calculate wheel circumference (i.e., diameter * pi) and a cumulative log of distance (i.e.,
distance = distance + circumference every time the wheel spun round by one revolution. Instantaneous speed was calculated
by taking the millis() value at the time the reed switch was triggered last time and subtracting this from the current value,
then dividing the wheel circumference by this difference in time. Top speed was simply determined by
if (speednow > maxspeed) {maxspeed = speednow};. The ESP32's webpage at this point simply dumped the variables
reed switch state and distance travelled to its only response page.
17/12/24 - Adding a motion sensor
Using an arduino-compatible motion sensor (such as this one),
a second variable, i.e., tracking movement, was possible. The sensor goes high when it detects movement, and drops to low again when movement ceases.
The variable motionCount was created to simply log the number of times motion was detected, i.e.,
motionCount = motionCount++; whenever if (digitalRead(pinMotion)) became true.
18/12/24 - Updating the server outputs to make an API
Now the rudimentary steps were understood, it was time to take things further and enable a more human interface for the ESP32,
as well as the ability to poll it remotely. It wouldn't be sensible to have it accessed straight through a firewall via an open port,
considering the potential for easy hacking, so polling data from it via a server call i.e., http://esp32.local/d/motionCount
meant that a more secure device, such as a Raspberry Pi or even NAS, could query it and display the data sensibly. So the following variables
were created to allow this:
distance - total distance travelled, i.e., motionCount * wheelCircumference
maxspeed - as described above
avespeed - total distance / time spent moving (i.e., ignoring gaps > 10 seconds)
millisnow - current boot epoch time, an unsigned long, for the ESP32, in milliseconds
lastwheelmillis - value of millis() last time a wheel spin was detected
lastmotionmillis - ditto for movement being sensed
motioncount - number of times motion detected
19/12/24 - Introducing PYTHON and the Raspberry Pi
It now became highly necessary to not just display the data from the chip in a sensible manner, using PHP calls from a
Raspberry Pi running Apache 3 and PHP, but also I thought it would be nice to have a bit more granular data,
not just knowing total distances travelled and maximum speeds, but the ability to scroll back through time and see
how the data has changed. Thus, Python was invoked, to regularly poll data from the ESP32 and store the variables below into a .csv file:
distance
motionCount
Python (a surprisingly pleasant programming language to use following the data type-pernickety C++), was able to simply loop through its
polling code,
either indefinitely or for a certain number of repetitions, as defined by input arguments to the program. It wasn't necessary to compile
the script to run stand-alone as it doesn't need to be real-time or processor-intensive, simply pinging the ESP32 every 30 seconds or so
is enough to generate a fairly large .CSV file.
20/12/24 - Making a lovely front-end
This webpage is the end-result of generating the CSV file mentioned above, and parsing it out to a "user-friendly" webpage - which is
still a work-in-progress of course. The CSV file is taken by PHP server-side, and broken down into individual rows of the array $data[],
using the following code,
Each line of the resulting array is then checked to see if it matches the line above, and if so, ignored. At present the data is simply pumped
out into a tabular format using good old fashioned HTML tables, but the plan going forward is to make this data table adjustable in JavaScript
to allow a specific time windows, and also eventually to make a vector graphics (SVG) graph of time vs. distance travelled, motion triggers, and possibly
even running speed (over the captured data polled windows of 30-45 seconds). As of the time of writing, I've just about managed to make a sine wave out of the
SVG with a bit of text overlay, and the data is in very long scrolling table.
22/12/24 - Unexpected spanner in the works
A very excitable wife and children have resulted in Diesel the hamster being gifted a three-layer house, instead of the bungalow cage
in which he was inhabiting up until today. His new mansion now has two wheels, slides, and an overhanging sleeping area. This means I'm going
to have to adapt my ESP32 wiring and code to account for THREE motion sensors, and TWO reed switches, as opposed to simply one of each!
Amazon orders have been made...
23/12/24 - Three movement sensors and two wheel sensors
The challenge accepted - and thankfully renaming a few variables and (perhaps lazily) copying some code to define if() loops for each wheel and each
level of the cage has resulted in being able to output several variables where once lay only two, namely, distance1 and distance2, and motion1count,
motion2count and motion3count. The info logger on the Raspberry Pi needed updating to record five variables instead of three, and additional variables
wheelNumberLast and motionLevelLast were created so that the output can reflect which wheel was used last and which level Diesel was last seen on.
28/12/24 - Javascript Chart.js updates
A fair amount of work went into the client-side behind-the-scenes JavaScript today, with the abandonment of the original static SVG-based chart output in preference
for an includable publicly-available JavaScript library called chart.js,
"one of the simplest visualization libraries for JavaScript, and comes with the many built-in chart types". There's a small amount of server-side work in the
PHP file, to generate five XY lists for the scatterplot, which annoyingly cannot accept simply array columns but must be in the format "x: 000, y: 000";
ideally it'd be nice to just have a selector enabling or disabling array column calls (i.e., dataarray[0] vs dataarray[4], but that seems not to be possible.
29/12/24 - Javascript Chart.js updates
More front-end updating behind the scenes, now the chart has three lines and is over a date axis rather than UNIX time; also, a legend now exists. The table output
has also had a spruce-up with cell background colours reflecting the cell values (via incorporating the formula
((250-$contrast)+(($maxMotion1abs - $data[$row][4])/$maxMotion2abs)*$contrast) within the style="background-color:rgb(r,g,b);" element.
31/12/24 - Events become seconds
The motion sensors have two different output modes, depending on the position of Jumper J1 (the external-most setting being signal ON trigger,
the inner-most setting being signal WHILE triggered). A re-write of the ESP32 C++ code to summate the time taken while triggered (float), as opposed to just incrementing
a trigger count (integer), I feel makes for more granular data. The sensors aren't the world's most reliable methods of determining fine-detected movement, as they
have only a limited field of view (especially in something so small as a hamster cage), and also on cessation of movement detection, require three seconds to be
reactivated. However, if Diesel keeps moving in front of their fields of view during this time, his movement should be recorded in seconds (to 2 decimal places). The changed code
was as follows, to accommodate seconds instead of just a simple count:
Hurrah! After quite a long wait, I had a brainwave while trying to measure how far the reed switch (basically a magnetic on/off sensor)
would need to protrude into the cage. I had initially thought I'd 3D print something to lie alongside his wheel, anchored from the outside of the cage, and have the reed switch slide in to. But in measuring the wheel with a biro, it hit me: use a biro! So 10 minutes in the garage cutting 5mm thick slivers of 2x2 pine and drilling four holes for cable ties and one hole for the biro, and sawing the tube of a biro to length, and we now have a working wheel sensor. Plugged it into the breadboard and she worked off the bat!
6/01/25 - Daily Logs now available!
A bit of back-end fiddling today, I have updated the Python script to use outfile = datetime.now().strftime("%Y") + datetime.now().strftime("%m") + datetime.now().strftime("%d") + ".csv" instead of a standard "datalogger.csv" filename,
thereby allowing daily logs to be generated on the Raspberry Pi, changing name at midnight. The Pi's webserver now has a simple script simply to find .csv files within the logging directory, which in turn is called by the front-end webpage and presented as a drop-down selection box to choose which logfile you want to view.
7/01/25 - Daily reset of variables
Just a quick one. Tried to figure out the best way to do allow the logfiles and subsequent information
to display data fresh for each day. I thought about rewriting the python code in the Raspberry Pi
to take the difference between two polled readings from the ESP32, and even rewriting the ESP32
code to simply output said difference between each access, but quickly realised the
workload doing this would involve re-writing the scripts for every bit of the data analysis side
(i.e., scripts for the graphs, tables, and csv file generator) AND refashioning each logfile to incorporate
the change to the parsers. Instead, it seems obvious now: at time the value of the current hour is
LESS than the previously stored value, i.e., 0 vs 23 then call the RESET programme from the ESP32. In the case of
clocks going back, because they only do this by one hour, this should not fail (as 01:59:59 rolls over to 01:00:00).
The limitation to this method is that of power failure, whereupon the ESP32 chip will reset all its data
back to 0, so a step-change would be noted within the CSV data, and an according severe negative shift be noted in the table and
presented data. I will amend the parser to recognise if a negative result occurs, and simply add on the difference to subsequent
lines.
13/01/25 - Re-jigging of the log files and enabling daily digests
Well I'm sorry it's been a while - though I somewhat doubt there's anyone actually reading this but if you are there, please drop me a line using
the main website Comments page (click here!). I've been letting the logs build up a bit, with keen interest,
and notice that Diesel is indeed using both wheels, though preferring his upper one (away from his sleeping quarters), and is active once every
three hours of so day or night - which is interesting. I've been editing the logfiles so that they are in keeping with the "up to date" standard
of the logfiles now, i.e., changing filename at midnight, and providing only a cumulative log from 0 each day. Via the addition of the following
code within the main Raspberry Pi python script, a new file is also being made which saves Diesel's total efforts of the day into a summary logfile,
though I haven't yet written a parser for this.
28/01/25 - Data analysis templates updated, logfile selector updated
Changed the way the log file selector works on the Data Table / Graph page, so now the drop-down shows the dd/mm/yyyy result of the logfile name
instead of its raw csv file value. Also allows the inclusion of longtermlog.csv to be referenced, and in a human-friendly way. Behind the scenes
the logfiles are still called via their filenames and passed to the datatable.php page. Graph programming was a bit tricky as the original
parser script was designed for cumulative data rather than absolute, but this was tweaked using an if/else statement in the code, i.e.,
if ( $logfile = "longtermlog.csv" ) { parse as non-cumulative } else { parse as cumulative }; .
Also now edited is the basicdata.php file, which is now able to summate the daily log values from the longtermlog.csv file
enabling long-term totals of data since logging began. Turns out that little Diesel has run quite a long way since the early days: 2.3km and counting!
Plans for the future
The following will be updated as more progress is made...
Making a suitable case for the veroboard instead of having the breadboards just sitting below Diesel's cage
