Project #1: Eat It

Eat It is an Augmented Reality app, developed using Vuforia and Unity.

Demo Video

Please watch the following video for a quick demonstration of the app and its features:

Downloads

View project on GitHub

Download Android App

Download Image Targets

Table of contents

  1. Demo Video
  2. Downloads
  3. Table of contents
  4. How to use
  5. Features
  6. Tools Used
  7. Screenshots
  8. Notes on Development
  9. Assets used
    1. Placemat 1 Scene
    2. Cereal Box 1 Scene
    3. Cereal Box 2 Scene
    4. Drink 1 Scene
    5. Apple Jacks Scene
    6. Magazine Scene

How to use

The Android version of this app can be downloaded from its GitHub releases page.

Alternatively, you can download the source code from GitHub:

  1. Download the zip file from master branch
  2. Unzip the contents and browse to the extracted folder
  3. Install Unity, if you don’t have it installed already
  4. Now browse to Assets => Scenes and open the MainScene.unity file using Unity
  5. Switch on your webcam and hit the play button

You will also need to print the image targets to use with the app. These targets can be downloaded from here.

Features

Tools Used

Screenshots

Props to be used as targets for the app Props to be used as targets for the app

3D Scene around the Placemat 3D Scene around the Placemat

Animated clouds in the Placemat Scene Animated clouds in the Placemat Scene

3D Scene around the Drink target 3D Scene around the Drink target

3D Scene with a honey flavored cereal theme, around the Cereal Box 1 Target 3D Scene around the Cereal Box 1 Target

3D Scene around the Cereal Box 1 Target (Zoomed Out) 3D Scene around the Cereal Box 1 Target (Zoomed Out)

3D Scene with a Sci-Fi theme, around the Cereal Box 2 Target 3D Scene with a Sci-Fi theme, around the Cereal Box 2 Target

Hovering Air Drone on top of the building Hovering Air Drone on top of the building

3D Scene around an Apple Jacks cereal box, as a real world example of this technology 3D Scene around an Apple Jacks cereal box, as a real world example of this technology

3D Magazine whose pages can be flipped with an animation using the virtual buttons 3D Magazine whose pages can be flipped with an animation using the virtual buttons

Side by side – Cereal Box 1 and Drink Side by side – Cereal Box 1 and Drink Honey Bees coming out of their Bee-Hive and the telekinetic cow causing an earthquake, when the props are placed close enough Honey Bees coming out of their Bee-Hive and the telekinetic cow causing an earthquake, when the props are placed close enough

Side by side – Apple Jacks and Cereal Box 2 Side by side – Apple Jacks and Cereal Box 2

Apples falling from the apple tree and alien mushrooms grow when the two props are placed close to each other Apples falling from the apple tree and alien mushrooms grow when the two props are placed close to each other

Apple Jacks showing its nutritional information when placed on the placemat Apple Jacks showing its nutritional information when placed on the placemat

Cereal Box 2 showing its nutritional information when placed on the placemat Cereal Box 2 showing its nutritional information when placed on the placemat

Milk Bottle showing its nutritional information when placed on the placemat Milk Bottle showing its nutritional information when placed on the placemat

Cereal Box 1 showing its nutritional information when placed on the placemat; Notice the Placemat's mascot expressing his opinion whenever any of the drinks or boxes are placed on the placemat Cereal Box 1 showing its nutritional information when placed on the placemat; Notice the Placemat’s mascot expressing his opinion whenever any of the drinks or boxes are placed on the placemat

Notes on Development

Stage 1 – The Noob

I found the project very challenging. It was the first time I was developing an AR application, but working with Vuforia and Unity made it very simple. With the help of Unity and Vuforia, you can easily display a scene on or around your real objects without wrtiting a single line of code!

Initially, I ended up spending most of my time to understand how to work with Unity and finding some good websites where I can find good quality (and free) assets to use in my scene. I found that SketchFab is a very good resource to search for any kind of 3D model. You will notice that most of my 3D models have been downloaded from this website. Most of the assets that I downloaded from other websites, didn’t work for me. Most of the times they will be missing the required Materials, or they will be in a proprietary format, such as .max (3DS Max) and .maya (Maya), which can only be opened if I had access to their softwares (which I clearly didn’t have).

Upon importing an asset, most of the times they don’t work out of the box. You need to manually map the materials with the provided textures, and play with the material’s settings, to make it resemble the original object as depicted on the artist’s render of the object.

Stage 2 – Trying to create cool stuff using Blender

I couldn’t find any good, free resources for some of the objects that I wanted to put in my scenes, so I tried to create them on my own. At first, I tried SketchUp, but found the features of the free version to be very limited, and I didn’t like the idea of working inside a browser as well. Hence, I spent a fair share of my day learning Blender, which, at first, was very confusing to use – who uses right-click as a primary button!! But, I soon realized how powerful it was. I found the Introduction to Blender series of tutorials, by Brackeys very helpful to get started with Blender, and it’s probably the best tutorial I ever came across on YouTube.

Using Blender, I created a Milk Bottle for the Drink 1 Scene, and used Photoshop to create a simple texture to brand the milk bottle as Legendairy Milk. The image of the cow was taken from the 3D model I used for the cow.

Next, I created a 3D model for a Bee Hive. I followed this tutorial by pauljs75, to get started with creating a honeycomb pattern in blender, but I had a hard time trying to make it look like a bee hive. For the sake of my sanity, I finally had to give up trying to create a perfect bee hive and settle for something which I can pass off as one.

I also had to use Blender for fixing some intersecting polygons in the model that I used for Cheese, since Unity was happily not rendering half of the upper face of the cheese. For the building in the Cereal Box 2 scene (the Sci-Fi one), I had to chop off all the buildings that came with the 3D model that I downloaded, except the one that I wanted. Luckily, for both these models, the original artist had provided the asset in .blend format.

Stage 3 – Adding Animation

Unity’s animation engine is very powerful, but lacks a very important feature – if you animate the properties of some objects in your scene, and then decide to change its hierarchy, to better organize your scene, you will end up breaking the references to those objects, and consequently, your animation.

Some of the models that I used came with an animation clip, such as:

I just had to create an Animator Controller for the above, and add these animation clips to the controller as the Default state.

All the other animations were done by me, and were orchestrated with the help of state machines, layers and parameters in the Animator Controller. Here is a list of animations that I added to my scenes:

Stage 4 – Adding Interactivity and Encountering Bugs

Till now, all the scenes were just there, whenever an object came in the camera’s view. The next step was to add interactivity.

To detect when two objects were close to each other, I added Colliders to all my scenes with the Is Trigger option checked. I also added the Rigid body component to these scenes so that my scripts may receive the OnTriggerEnter and the OnTriggerExit events whenever collisions are detected. The script will then check which object it collided with, and appropriately trigger the animations. If the objects collide with the placemat, their corresponding scenes will be hidden and their nutritional information will be shown instead (all the nutritional information was sourced from nutritionix.com).

During testing, I found that my cereal boxes were going halfway inside the placemat, and when placed side by side, one would appear to be placed closer to the camera as compared to the other. I spent a couple of hours trying out different things to fix the issue, without any success. Finally, I decided to measure the physical size of all my props, and re-upload all the target images to Vuforia Target Manager and download the new database.

Updating my database with a new one, messed up all of my scenes, and I had to rescale and position everything again. I ended up wasting a full day due to this, but thankfully it solved the issue!

Stage 5 – Designing the Magazine and Lighting the scenes

First step was to create the animation and code the interactivity. I wanted to have a 3D page turn animation whenever the user touched (or rather, obstructed) the virtual buttons. After searching and trying out different models for a book, I came across a 3D model that had a nice page flip animation. After importing it in Unity, I found that the model actually consisted of several pages, each acting like a keyframe in the different stages of a page flip animation. Hence, to animate the page flip, I created an animation clip that would hide and show the different pages. Now that the animation clip was ready, I wrote a script to appropriately change all the materials of these pages and trigger the animation whenever the virtual buttons were pressed.

The next step was to design the content. I decided to create a health magazine, consisting of a few healthy recipes for breakfast. I picked up a few recipes from Stacy Homemaker, Cooking Light, and Fitness Magazine, and designed the cover and the pages in Photoshop. Afterwards, it was just a simple matter of pointing my script to these images for the different pages.

For lighting my scenes, at first, I was using the default Directional Light, but they looked terrible with the same lighting configuration for all of them, especially, the Sci-Fi scene around Cereal Box 2, which had Emmisive materials for its 3D objects. So I removed the directional light and used 3 Point Lights for each scene, to effectively illuminate all parts of the scenes, and at the same time create a more warmer feel (as in the Honey themed Cereal Box 1 scene), or the more dark feel (as in the Sci Fi scene).

Stage 6 – The ill-fate of the Apple Jacks scene, and doing the submission

Due to shortage of time, I was not able to dedicate enough time to work on the additional requirements for Graduate students. So, I decided to use a box of Apple Jacks as my image target. I had initially bought the box to paste the cutouts for the two Cereal Box props (given to us by Professor Johnson), but the size didn’t match the prints and I wasn’t able to use them. Needless to say, the scene can use a lot of refinement as compared to the other scenes.

For the submission, I decided to build and run the app on my Android device, since I was experiencing some lag with my webcam and the video recording would be bad. I used AZ Screen Recorder app to record my phone’s screen, along with my narration while recording the video. I then looked for a decent video-editing software, and after trying out a few of them, I used VideoPad to put my video together, since none of the others were able to play the videos recorded from my phone. The last step was to upload the video on YouTube, and make this webpage.

Assets used

This project uses a lot of 3D models, textures and sound effects, which are freely available for non-commercial use. Here is a detailed list of all the models used and where I got them from.

The list is organized scene wise, and then according to the type of asset (3D object, Sound, Texture). The numbers on the image for each scene, indicates the S. No. of that object in the list.

Placemat 1 Scene

3D objects

Placemat 1 Scene

S. No. Name Source Artist License
1 Bread SketchFab andree Creative Commons Attribution
2 Pancakes SketchFab soidev Creative Commons Attribution
3 Plate SketchFab WinterPlayer Creative Commons Attribution
4 Rug SketchFab Nicholas Record Creative Commons Attribution
5 Clouds SketchFab PotatoBeenCrafted Creative Commons Attribution
6 Fruit Basket SketchFab John Greenstone Creative Commons Attribution
7 Chatter Teeth SketchFab Damco Creative Commons Attribution
8 Umbrella SketchFab Le.Manh.Cuong Creative Commons Attribution
9 Ground SketchFab Le.Manh.Cuong Creative Commons Attribution

Textures

S. No. Name Source Artist License
1 Grass 3D Textures João Paulo CCO 1.0 Universal - Public Domain Dedication

Sounds

S. No. Name Source Artist License
1 Nature Ambiance FreeSound blouhond Creative Commons Attribution 3.0
2 Mmmm PacDV PacDV Royalty Free
3 No PacDV PacDV Royalty Free
4 Woo Hoo PacDV PacDV Royalty Free
5 Ooh La La FreeSound blouhond Creative Commons Attribution 3.0

Cereal Box 1 Scene

3D objects

Cereal Box 1 Scene

S. No. Name Source Artist License
1 Bee Hive - Mayank K Rastogi -
2 Bee TurboSquid can ocak Royalty Free License - All Extended Uses
3 Green Tree SketchFab MaximilianY. Creative Commons Attribution
4 Birch Tree SketchFab Alan Zimmerman Creative Commons Attribution - Non-commercial
5 Colored Flowers SketchFab tojamerlin Creative Commons Attribution
6 White Flowers SketchFab tojamerlin Creative Commons Attribution
7 The Wasp SketchFab Hector Mora Creative Commons Attribution
8 Ground SketchFab Le.Manh.Cuong Creative Commons Attribution

Textures

S. No. Name Source Artist License
1 Grass with Flowers 3D Textures João Paulo CCO 1.0 Universal - Public Domain Dedication

Sounds

S. No. Name Source Artist License
1 Bee Buzzing SalamiSound SalamiSound Free for non-commercial use

Cereal Box 2 Scene

3D objects

Cereal Box 2 Scene

S. No. Name Source Artist License
1 Air Drone SketchFab Jason.Arens Creative Commons Attribution
2 Buster Drone SketchFab LaVADraGoN Creative Commons Attribution - Non-commercial
3 Sci-Fi Plants SketchFab DarkLordFlash Creative Commons Attribution
4 Building SketchFab d880 Creative Commons Attribution
5 Helipad SketchFab stefan89 Creative Commons Attribution
6 Ground SketchFab Le.Manh.Cuong Creative Commons Attribution

Textures

S. No. Name Source Artist License
1 Terrain 3D Textures João Paulo CCO 1.0 Universal - Public Domain Dedication

Sounds

S. No. Name Source Artist License
1 Sci-Fi Ambiance SoundBible Daniel Simion Creative Commons Attribution 3.0

Drink 1 Scene

3D objects

Drink 1 Scene

S. No. Name Source Artist License
1 Milk Bottle - Mayank K Rastogi -
2 Coffee Maker SketchFab amrmosho Creative Commons Attribution
3 Plate SketchFab WinterPlayer Creative Commons Attribution
4 Cheese SketchFab Christian Gentry Creative Commons Attribution
5 Coffee Mug SketchFab French Baguette Creative Commons Attribution
6 Teapot SketchFab WinterPlayer Creative Commons Attribution - Non-Commercial
7 Cow SketchFab shakiller Creative Commons Attribution
8 Counter-top 3D Textures João Paulo CCO 1.0 Universal - Public Domain Dedication

Textures

S. No. Name Source Artist License
1 White Marble 3D Textures João Paulo CCO 1.0 Universal - Public Domain Dedication

Sounds

S. No. Name Source Artist License
1 Cow Moo SoundBible BuffBill84 Creative Commons Attribution 3.0

Apple Jacks Scene

3D objects

AppleJacks 1 Scene

S. No. Name Source Artist License
1 Cereal Bowl TurboSquid cgs3d Royalty Free License - All Extended Uses
2 Green Tree SketchFab MaximilianY. Creative Commons Attribution
2 Yellow(ish) Tree SketchFab MaximilianY. Creative Commons Attribution
4 Ground SketchFab Le.Manh.Cuong Creative Commons Attribution

Textures

S. No. Name Source Artist License
1 Grass 3D Textures João Paulo CCO 1.0 Universal - Public Domain Dedication
2 Apple SketchFab Le.Manh.Cuong Creative Commons Attribution

Magazine Scene

3D objects

Magazine Scene

S. No. Name Source Artist License
1 Book SketchFab John Fino Creative Commons Attribution

Sounds

S. No. Name Source Artist License
1 Page Flip Sound Jay Sound Jay Royalty Free

[Edit] Oct 3, 2018: Updated instructions to open the “MainScene.unity” scene after downloading the project; Added table of contents