ABOUT PRESENTATION

So far in our Analysis & Design of Technology-Based Learning Environments class (EDIT732), we've been talking about Augmented Reality (AR) apps from a consumer's perspective and the instructional design challenges therein.  My presentation is going to shed light on the technical aspects of AR apps and how to use App Inventor to create them.  This presentation may be a bit longer than other class presentations as it expects you to use your computer to perform the various setup, installation, and programming steps (tutorials) needed to become familiar with using App Inventor.  These steps will ultimately teach you how to use App Inventor to manipulate the technical features of a device, in order to create any augmented reality app you can think of.... and any other app for that matter.  I will 'virtually' pause to give you time to perform these steps and then 'virtually' resume again, presuming you have successfully completed them.

Let's begin.

AUGMENTED REALITY APPS (briefly explained)

Technically speaking, what is an augmented reality app, anyway?


Most Augmented Reality (AR) Mobile Apps are designed to overlay location data (points, images, etc.) onto the live view of a mobile device's camera. In a sense, AR apps allow your phone to become a window into a data-driven world.

To do this, AR apps are enabled through the use of a smartphone's camera and its sensors - these being  the location sensor, which uses GPS - the Global Positioning System administered by the US; the orientation sensor, which detects in which direction your device is pointing; and the accelerometer, a sensor that detects and reports movement and acceleration in 3 directions (x, y, z). It is important to remember that these features are detected in the device only, not the person holding the device.  If a person moves and leaves his/her device stationary, nothing will happen.  Essentially, this means, augmented reality apps require a device.

In this presentation, I will introduce App Inventor, a mobile application development tool created to help nonprogrammers build a wide variety of apps for android devices.  Although introducing you to the entire platform is necessary for app development, the main goal of this presentation is to highlight those features of the tool that are used to create augmented reality apps - namely those programming elements that control a smartphone's camera and the sensors I just described.  But more on those later. 

For now, let me share how this presentation is structured and how it will help you learn about the programming features needed to create an AR app.

DESIGN AND USE OF THIS PRESENTATION

App Inventor has been around for over three years.  It was first created at Google and is now hosted and maintained by the Center for Mobile Learning at MIT.  As a result, there are numerous resources and introductory tutorials available online and in printed form to help you get acquainted with how to use it.  Therefore, to not recreate the wheel, I will simply give an overview of these features and point you to these resources as we begin.  I've worked directly with the App Inventor Team for over two years now and am fairly confident in the material available to give newcomers a tour.  As such, I'll point you to the installation and setup processes of App Inventor as well as resources that can help you troubleshoot any issues you may have.


Later, you can follow some of these resources and the tutorials they feature to guide you through the creation of your own android apps.  It is my hope that, overall, these activities will demonstrate how to use App Inventor,  showcase its components, and illustrate how they are used to create an app.

Once done, we will drill down into the components that enable the creation of AR apps.  We will watch a few examples and take a quiz to test your knowledge.  OK, I'm just kidding about the quiz.  To conclude the presentation, more instructional resources will be shared along with other platforms that can also be used to build AR apps.

With that,  here's a bit of background information.


INTRODUCING APP INVENTOR FOR ANDROID

As previous mentioned, App Inventor is hosted by MIT.  It was created by MIT Professor Hal Abelson, while he was a visiting professor at Google.  It can be considered a younger sister of the Scratch programming language, created at MIT, as it has adopted many factors from Scratch.  App Inventor and Scratch are both visual programming languages.  That means, they enable users to create programs using visual block-like puzzle pieces.  Programmer connect and stack these programming blocks in the same way that someone would stack lego pieces.


Like Scratch blocks, App Inventor blocks are categorized to easily identify which blocks are needed to build the specific features of a particular app.  This is one of the many reasons why App Inventor caters to people who want to build apps but are not professional software developers.  There is no text-based code involved and very little programming knowledge needed.  Any programming concepts that are helpful, will be explained here.   I presume many of you fit into this nonprogrammer category (and even if you don't), so let's get started.

App Inventor helps you create apps with a web-browser and an android phone or on-screen phone emulator.

You must also have a Google Gmail account to access and use App Inventor.  If you do not have a gmail account, please pause the audio here to create one.

[virtual pause]

All things App Inventor can be found on its website - http://appinventor.mit.edu.  Please take a moment to point your browser there now. Try to view the page while still listening, if feasible. 

[virtual pause, if needed]

The website caters to educators, those who simply want to explore its features and capabilities, and to programmers.  These audiences are represented by the 3 main buttons on the homepage - Teach, Explore, Invent.   For starters, we will focus on Explore and Invent.  I will use some of the resources in the Teach section to walk you through getting setup.  Additional resources from the Teach section will be shared throughout presentation.

For now, Click Explore.  Before exploring the 8 tabs across the top, I'd like you to take at minute to watch the introductory video on the left.  It features and older version of App Inventor, but watching it should give you an idea of what you are in for....  I've included the video below as well. If you watch it on the site, feel free to skip it here.

I'll pause here while you watch.

[virtual pause]

Pretty cool, huh? 

APP INVENTOR: DESIGNER, BLOCKS EDITOR, EMULATOR

As you've seen in the video, there are 3 components to App Inventor that allow you to build apps:

  1. The Designer
  2. The Blocks Editor
  3. Android Emulator
  4. Android Phone (optional)

 

The designer is used to indicate the components needed in your app and to set their properties (or values), like the color of a button or the text of a label.  The actions you take- drag and drop - are very similar to creating a slideshow presentation. One the left is the palette from which you choose your components (categorized by type), in the middle is the viewer (so you can see what you are designing), and the right features the various properties for each component selected and allows you to set initial property values (i.e. color, size, text for label, etc.) 


[click image for larger view]

The Blocks Editor allows you to stack the blocks associated with the components you used in the designer to create stackable elements of code.  Taken together, these stacks execute as your app. You have 'built-in' blocks that control math equations, flow of control (i.e repeat or if-then scenarios), while the 'my blocks' section holds the blocks associated with those components you used in the designer.


[click image for larger view]

And the Android Emulator allows your computer to act like an android device so that you can test your app as it is being developed.  This is especially useful if you do not have an android device.


[click image for larger view]

Now let's walk through the setup process so you can learn how to create an app yourself.   At the time of this presentation, App Inventor successfully runs on Windows, Mac OS, and Linux machines.

As we briefly go through the setup and installation processes, please remember there are many resources on the website and on Youtube that will help you with installation, setup, troubleshooting, and building some pretty basic but powerful apps.

First, you will need to ensure that your computer is ready to run App Inventor and its components.  To do this, please click the SETUP Tab in the Explore section of the App Inventor website and follow the steps indicated.  You should have opened this website in a previous step above. If you'd like more detailed step-by-step instructions, please follow those featured on this page - http://appinventor.mit.edu/teach/curriculum/setup-instructions.html

I'll wait here while you get your machine ready.  Feel free to pause the audio while you do this. When you are done, you can go through a few tutorials as the instructions suggest.  Either way, you will be ready to build an app.  So, click on the Invent button and resume this presentation when you are ready. This will start App Inventor.

[virtual pause]

Now that you're back we are ready to start App Inventor. Here are a few videos that will guide you through starting a new project and creating your first app.  While doing so, you will be given a tour of the designer, the blocks editor, and the emulator. If you feel you understand what the designer is from the first clip, you can skip the 2nd. The 2nd clip is here for reinforcement.

Detailed instructions, images, and audio files for this Hello Purr tutorial can be found at http://appinventor.mit.edu/explore/content/hellopurr-emulator-part-1.html

[virtual pause]


[Designer]


[Designer]


[Blocks Editor & Emulator]

As an aside, Whenever you are building an app, you should simply keep your emulator open on the side, so you don't have to always wait for it to launch.  If you have an android device, you can also connect it to your computer and watch your changes live, as you work.

PROPERTIES, BLOCKS, EVENTS, METHODS/COMMANDS

Now that you have been introduced to App Inventor, let's review some programming concepts and add more meaning to these blocks.

The Blocks Editor is the heart of app creation with App Inventor and the App Inventor team has created blocks for just about everything you can do with an Android device as well as blocks for doing "programming-like" stuff-- blocks to store information, blocks for repeating actions, and blocks to perform actions under certain conditions. There are even blocks to talk to services like Twitter.

These blocks combine to form the making of a functioning app. Even though you can set the initial properties of the components of your app in the designer, you can use the Blocks Editor to change them throughout the execution of your app.  When a component changes in some way, these are called behaviors.  Another way to think of behaviors is as a method (a sequence of steps, if you will) executed in order to make a change.  These methods either 'get' or 'set' a component's property or value.  Methods are fired by events, and events are actions taken by a user.

For instance, when a user taps a button (an event), the buttons background color (its property) can change to another color (like red, which is a value) using the button's set 'button.backgroundcolor block' (command/method).

When this happens, the app is commanded to play a sound.  The image of that button and the target sound to play were originally assigned as properties in the designer.

When Button Clicked Play Sound Blocks

COMPONENTS, PROPERTIES, BLOCKS, EVENTS

Visit http://blog.ouseful.info/2010/07/20/overview-of-google-app-inventor-blocks/ for an overview of App Inventor blocks, their accompanying 'set' commands, and the blocks used when a 'get' is issued for a value. The illustrations and colors are from an older versions of the App Inventor but, for the most part, the current blocks have the same shapes and meanings.

[virtual pause]

Combining these blocks with more details about their corresponding components, their properties, events, and methods provides a better understanding of how App Inventor helps you build apps without the need to know how to write code. Remember, each component in the Palette on the left has one or more events associated with it.  For a detailed view of each component, its changeable properties, and its corresponding events, please point your browser to - http://appinventor.mit.edu/explore/content/reference-documentation.html

[virtual pause]

*** PLAN YOUR APP BEFORE BEGINNING TO BUILD WITH APP INVENTOR! ***

IT IS IMPERATIVE THAT YOU PLAN YOUR APP, IDEATION, SKETCHES OF EACH SCREEN AND/OR ACTION, AND ANY PARTICULAR BLOCKS NEEDED, bEFORE YOU START BUILDING.  THIS WILL SAVE YOU TIME AND FRUSTRATION.  IT MAY TAKE YOU A WHILE TO GET FAMILIAR WITH THESE ELEMENTS, BUT, AS WITH ANYTHING, THE MORE YOU DO IT, THE BETTER YOU BECOME. THE TWO RESOURCES ABOVE WILL BE INVALUABLE.

Let's continue.

With these combined notions (components, properties, blocks, events, etc.), you can now build and understand any tutorial on the App Inventor website.... and use them to build your own apps - AR or otherwise.

AUGMENTED REALITY APP DEVELOPMENT with CAMERA & SENSORS

Speaking of AR, let's now zero in on the components used in AR apps.  Remember these are the camera,  the location sensor, the orientation sensor, and the accelerometer sensor.  They are found in the Designer Palette in the Media and Sensors categories respectively. 

I presume you already know what a device's camera does, so I won't explain it here.  But feel free to follow this tutorial that teaches you how to write an app to launch your device's camera, and take and save a picture.

[virtual pause]


[camera app tutorial]

Now, on to the sensors. Here is some general information about the data these sensors provide and some tutorials on how to implement them in your app.

Longitude -  (how far east or west from the prime meridian) = positive and negative values respectively -180 to 180
Latitude -  (how far north or south you are from the equator) =positive and negative values respectively -90 to 90
Altitude - (basically means the height from a given point, but can sort of complicated to explain in context of gps, so I'll suggest you google it for your own understanding of geometric altitude)

The other measurements help determine which way is north, so you can easily determine south, east, and west.

Below are some sensor-related tutorials and readings.  Feel free to pause this recording in order to follow them.

Location Sensor  & Orientation Sensor - http://www.cs.trincoll.edu/~ram/cpsc110/tutorials/whereisnorth/hw-whereisnorth.html

Accelerometer Sensor -

GPS - https://sites.google.com/site/blocks123/gps

You can also read Chapter 23: Reading and Responding to Sensors (8 pgs) in the App Inventor Create Your Own Android Apps book written by members of the App Inventor Team. http://cs.usfca.edu/~wolber/appinventor/bookSplits/ch23Sensors.pdf

[virtual pause]

AUGMENTED REALITY TUTORIALS

Now that you have been introduced to the components that make augmented reality possible in most apps, here are a few example apps and some tutorials to show them in action.

Currently there is a very popular augmented reality app created with App Inventor.  It's called Draught and is an augmented reality game that lets users play checkers using GPS to track their location and movement.
http://appinventor.mit.edu/explore/stories/draughts-game-virtual-reality-app-inventor.html

Here's another set of tutorials that focuses on the Wikitude, Google Sky Map, and Amazon Bookstore Browser AR apps - https://sites.google.com/site/appinventor/augmented-reality

[virtual pause]

OTHER PLATFORMS

In addition to App Inventor, there are other fairly well-known platforms available that will help create augmented reality apps.  Here is a brief list with URLS so you can investigate them for yourself:

 

IMPLICATIONS for INSTRUCTIONAL DESIGN - SOFTWARE DEVELOPMENT SKILLS A MUST

So, what does all this mean for Instructional Designers, especially those who may want to embark on building protypes for mobile learning apps, even augmented reality mobile learning apps? Well, since I've been attending instructional design classes here at GMU (just a few semesters), I have come to realize that emerging technologies and the skills needed to implement them call for the integration of more computer scientists in the instructional design process - software development skills at the very least. When the best of all emerging technologies (hardware, software, web) are expected to be implemented when creating an optimal learning environment, I think it is imperative for at least one software developer to be on the team. I am finding that it is rare for someone to be an instructional designer and a software developer. Tools like App Inventor and others mentioned below help bridge this gap. At the very least, it allows for everyone to 'speak' the same language - or at least be on the same page.


There you have it.  I hope you enjoyed this introduction to App Inventor and I wish you much success in becoming an app developer - augmented reality or not.

 

Additional Development Resources (App Inventor and otherwise):


Other app development platforms
-If you know HTML, CSS, Javascript (Web designers' toolkit), you can use PhoneGap (recently purchased by Adobe) or Appcelerate.

 

Thanks for your time and attention.  Please feel free to contact me with any questions.

 

CREDITS:

The audio for this presentation was created and edited with Audacity, free, open-source, cross-platform software for recording and editing sounds.

Most images used in this presentation were obtained from the App Inventor website and other websites, as indicated.

Title Images from:
iOnRoad - http://reviews.cnet.com/8301-13746_7-20067732-48.html
Layor - http://wirelessandmobilenews.com/2009/08/android-apps-2day-layar-reality-browser-adds-info-to-photos-where-u-at.html