🌍 https://ucl-casa-ce.github.io/artes
ARTES uses WebAR and physical markers to visualise real time data from MQTT feeds. It has been first developed during the EAST SUMMER SCHOOL 2021, 26 July – 6 August 2021
Discover how to build a simple soil sensor prototype to make sure your plants are getting the right amount of light and water, and design an AR (augmented reality) marker using mosaic tiles. In this hands-on creative tech space, you will learn how to make your marker come to life by making it interactive, pulling in data from the sensors, and accessing it via a smartphone. You will also have the opportunity to ask questions and find out more about the application of similar technologies in cutting-edge research projects by members of the Connected Environments Lab, such as the bat boxes in the Queen Elizabeth Olympic Park.
The same tips to create a custom AR.js marker applied to physical tiles. In our case we used Lego 2x2 flat tiles in 7 different colours:
- White
- Bright Red
- Bright Yellow
- Dark Green
- Bright Orange
- Medium Azur
- Earth Blue
The final marker uses a 3 by 3 tiles pattern (a 6 by 6 tiles pattern has been tested as well and it works just fine). The extra white tiles around the black frame are not essential but help in the detection of the marker.
Once the physical pattern is built, we need to translate it into a digital one. The makerTiles/tiles.html is a static (and very simple) page used to digitise and download an image of the tile made with the square lego elements (using html2canvas).
The saved images need to be square, file format PNG (with no alpha channel or transparency) or JPG. To match the colour of the physical LEGO tiles with their digital version we used the following HEX code (Thanks Christoph Bartneck for the great article):
| Tile | HEX |
|---|---|
| White | #FFFFFF |
| Bright Red | #ED1D2A |
| Bright Yellow | #FFD500 |
| Dark Green | #00843D |
| Bright Orange | #FF8200 |
| Medium Azur | #1D83AF |
| Earth Blue | #003865 |
During the event, we run this page using VSCode Live Server, and we used a touchscreen to easily select the tiles and their colours.
The folder ./markergen/node contains the scripts to create the pattern .patt used by AR.Js. It is based on the AR.js marker-training, practically the same code, but we did not have access to a reliable internet connection during the event so we used a slightly modified local version of it, that also allowed us to batch convert multiple images.
After installing the node_modules, using npm install, these are the steps to follow:
- copy the images to convert into Pattern file, the ones created using
makerTiles/tiles.html, in the ./markergen/node/img folder (remember, images need to be square, PNG -no alpha channel, no transparency- or JPG); - in a new
node.jsterminal, or in a new VSCode terminal, run the commandnode generatorfrom the ./markergen/node/node folder ;
The patterns will be created and saved in the folder ./markergen/node/patt;
NOTE: if you need also the digital format of the marker (with the black border) you can use the online tool, use a border ratio of 0.85
The patt files created with markergen/node/generator.js are used to trigger the WebAR experience and need to be moved to the resources/patt folder of the main website. We collected the real-time data from a series of sensors connected to Arduino boards and published to a MQTT broker (not covered here). We used MQTT.js to subscribe to the topics. All the WebAR logic is in markerManager.js while the 3D models of the plant and the screen are loaded from the index.html file. The markers are loaded using the marker.json file
{
"info": {
"version": "1.0",
"title": "WebAR-MultiMarkers",
"markers": 2
},
"content": [
{
"markerName": "pattern-image1",
"topic": "TOPIC/TO/SUBSCRIBE",
"subtopics": [
"temperature",
"humidity",
"moisture"
],
"type": "",
"textContent": "SAMPLE TEXT"
},
{
"markerName": "pattern-image2",
"topic": "TOPIC/TO/SUBSCRIBE",
"subtopics": [
"temperature",
"humidity",
"moisture"
],
"type": "",
"textContent": "SAMPLE TEXT"
}
]
}
NOTE: in order to make this project works, you will need to change some parameters in the
markerManager.js: the MQTT address as WebSocket with its port (generally 8080), line 32; the topics to subscribe to, line 70-71-72.
As WebAR needs to access the camera of the smartphone a secure HTTPS connection is needed, therefore also the Websocket connection needs to be secure (WSS). Easiest way is to use GitHub Pages or, for local development, something like npm package local-web-server as well as a self-signed certificate with VSCode.
Martin de Jode (UCL - CASA - Connected Environments)
Leah Lovett (UCL - CASA - Connected Environments) @leahlovett
Valerio Signorelli (UCL - CASA - Connected Environments) @ValeSignorelli