First let me start with some disclaimers to try to make sure this post is not misinterpreted: I am not arguing that the mouse and keyboard are really dead or that the lack of a mouse on tablets is a bad thing. I am not arguing that the ipad or similar devices are awful for education or content creation. I am just thinking about how to make them even better in these areas, and conceptual and technological roadblocks in the way. Some of the criticisms of the ipad as a content consumption device (here, here), have been addressed with the ipad 2 and other new android tablets with their inclusion of cameras and input ports, and some tablets are even coming out with a stylus, like the HTC Flyer.
But imagine any creative person – creating a 3d character for a game, drawing a picture, composing a music score, creating a graphically rich document or presentation, etc. They likely have something in their hand, or their hands are busy doing something. When that involves interacting with a computer they are likely clicking the mouse to drag something around or edit text, for example, or using a stylus on a digitizer surface (like a wacom) for drawing, or typing away on a keyboard.
These input devices are all essentially gone on new tablets and smartphones. You can still type (slower) on virtual keyboards, and you can click like a mouse with your finger (tap).
An example of the impact of this is rich text editing, like with a word or openoffice or google docs document. Many browser-based wysiwyg editing tools, which are used virtually everywhere, such as in moodle (which uses the TinyMCE editor) or drupal, no longer work when you access them from an ipad or iphone or android device (or other mobile platforms like blackberry or palm webos). Even the newest “HTML5″ editors, such as Aloha Editor, pop up an error message if you try to access them from a mobile device. Other browser-based editing and drawing tools also no longer work on these new platforms, or you have to draw with your fingers. Most of Google’s and others’ tools like Google Presentation do not work on mobile platforms. Really, just imagine most any software people use on a desktop to create stuff – like office, or the flash ide, or gimp/photoshop, blender 3d, etc. Even when programming, which really is just typing in plain text, we usually prefer to use IDEs that popup suggestions and corrections to help us out. For many of our desktop apps its hard to even imagine them working on a tablet or phone.
The rich text editing tools in browsers like TinyMCE or CKEditor primarily rely on the contenteditable HTML attribute to support editing. Add that attribute to an HTML element, and the contents of that element become editable inside the web browser. It works in all browsers, including old Internet Explorer versions. It doesn’t really work well or even at all on mobile browsers though (see here, here, here, here).
Newer versions of android, webkit, and mobile firefox have been slowly improving their support for contenteditable, and maybe they will eventually “fix” the issue, but I’m not sure that this will be fixed through engineering alone. Some code editors like codemirror 2 and the ace editor are trying out workarounds like using a hidden textfield that captures key presses. Codemirror 2 works on an ipad somewhat, the ace editor does not. It remains to be seen if a similar trick might work for a rich text editor (it’s tricky enough just to do it for plain text). And like I said, the HTC Flyer and other tablets (esp. those being designed for medical and other professionals) are starting to include a stylus, and it remains to be seen if that will catch on (it didn’t before with older tablets). Others are coming out with dual screen tablets, where the second touchscreen can work like a touchpad on a laptop or nintendo ds, but that also may not catch on.
Another more general alternative strategy to this issue of tablets having no mouse or other input devices other than the touchscreen and the camera (which can be used for gestural or other input), might be to conceptually rethink how to support multimedia creation on these mobile platforms. Perhaps we should drop the notion of “documents” or “pages”. After all, you don’t think of a flash widget as a page or document. You don’t think of a game as a set of pages or documents. And Apple and other developers have already created apps for some specialized types of content creation and creativity, such as musical instrument simulators and so forth.
So, this may be a pre-paradigmatic moment where we’ll see what catches on: will we try to perfectly “emulate” the mouse and stylus and its supported interactions via other means such as gestures, or will new and unique types of interactions continue to catch on (like multitouch stuff). Probably a combination of both, but so far the camera is hardly being used at all for input, other than recording videos or taking pictures. And some may dismiss the idea of a stylus ever catching on again, but Apple has occasionally made “mistakes” before (the first mac didn’t have a floppy drive, for example), and others have been successful in incorporating a stylus, like the Nintendo DS, which my little boy continually loses
Like I mentioned in the previous post, the CES show next week promises to reveal a slew of new android tablets and phones, so we’ll see what is announced (and what is actually released).
Here are some android tips & resources I’ve run across in the past year:
- How to root the Droid (the first droid)
- How to root the NookColor
- How to read library e-books on an android tablet – right now the only official way is to use Adobe Digital Editions (ADE, which only runs on windows or mac) and use a device that ADE supports, such as the NookColor or Pocket Edge. Here’s an ADE video tutorial.
- If you’ve got an older android phone or tablet with only a few screens, I’d recommend LauncherPro or a similar app to improve your home screens. See this comparison between LauncherPro and other options like ADWLauncher.
- Keeping up with android news: See this twitter list for various sources like androidcentral: http://twitter.com/#!/edtechdev/android
- Places to find recommended, popular apps
- Open source apps – see these lists of open source apps and open source games (you’ll have to google to find the actual source though).
- Android development
Some free educational apps:
- X Construction Lite – bridge building game
- Google Sky Map – shows the names of stars, planets, constellations that you point at, using the accelerometer and compass
- There are apps or android-friendly mobile front-ends for Blackboard (requires university subscription) and Moodle (see also http://mle.sourceforge.net/ and http://code.google.com/p/moodbile/)
- Video podcasting is possible with apps from Qik or Ustream viewer or broadcaster
- Physics: space physics lite, space simulator, physics calculator
- Musical instruments: mypiano, guitar solo lite
- Programming: IProgram, scripting layer for android, ruboto
- Various math practice apps like Math Workshop or Math Attack
- Of course there are all the e-book reader apps such as Aldiko, Nook, Kindle, …
- See also AppBrain’s education section and this list of 101 best android apps in education
- See also the IEAR site for many educational apps, but they are mostly iphone/ipad apps at this point.
Some free games:
- angry birds, of course
- hungry shark
- falling ball
- hit the penguin
- air attack
- my paper airplane
- toss it
- winds of steel
- flying high
For a couple of projects, I’m looking into tablets that students can use for taking notes and so forth. There are finally some android tablets out already or coming out this month. This google doc I created has some notes on upcoming android tablets, including prices and features. The Archos 101, for example, is comparable to the ipad but half the price. My conclusion at the moment however is to wait a bit longer until we hear more about the next versions of the android OS (2.3, 3.0), which will run much better on tablets. There seem to be issues with android 2.2 and tablets. We should hear more about android 2.3 and 3.0 very soon, and as the NVIDIA CEO says (the Nvidia Tegra dual-core processor is used in most next generation android tablets coming out), there are lots of android tablets coming out.
Android tablets do seem to be a major focus of hardware development now:
“This isn’t a fad. Everybody’s building tablets because it’s just so important. Car companies are working on tablets, consumer electronics companies are working on tablets, computer companies are working on tablets, and communications companies are working on tablets. The medical industry is working on tablets,” he said during the earnings conference call. “I don’t remember in the history of computing [when] a singular device is being worked on by all of the industry.”
Here’s a taste of the current tools and frameworks out there for developers interested in learning more about this platform. Probably the first thing you have to decide though, is are you more interested in running your HTML5 app on mobile phones and tablets (iPhone and iPad’s iOS, Android, and to some extent Blackberry and Palm), or in a regular desktop browser (Firefox, Chrome, Safari, IE…), or both? Some tools for mobile web app development are listed below.
First, here are some places to keep up with this rapidly evolving field:
- Learning WebGL – for keeping up with 3D developments
- Some discussion groups: http://groups.google.com/group/iphonewebdev , sitepoint forum, codingforums, etc.
- The Changelog – They have a regular podcast which often features interviews with developers of HTML tools such as some of the ones listed below (Sencha Touch, Coffeescript, etc.)
- YUI – very java-like user interface toolkit from Yahoo, very complete and accessible. They have started adding mobile support (touch/swipe/etc).
- JS Optimizers – to compress/obfuscate and combine into one file your js code see Google Closure Compiler, YUI Compressor, JSMIN
- PhoneGap – has an open source permissive license. They support Blackberry and Palm and Symbian and so forth, as well as iOS and Android. They create a native webview wrapper for your HTML5 app, so that you can access native things such as the camera or sensors.
- Sencha Touch – built off of ExtJS and JQTouch – any app you develop with it either needs to be GPL or else you have to pay for a commercial license from them. Can work with PhoneGap, they have some nice demos, esp. for the iPad.
- Ansca Mobile – Corona SDK, better for game development, commercial license.
- appMobi – commercial, too, I believe
- Canvas tutorials
- ExCanvas – for making canvas work in internet explorer (IE 9 will support it I believe)
- RGraph – graphing library
- bluff – graphing library
- three.js – canvas-based 3d engine (not using webgl as the ones below)
- Painting/Sketching demos
- Sass and Compass – alternative to CSS – adds some smarter features to CSS like variables and so forth
- HAML – alternate to HTML, not as popular perhaps, or as needed as the above two, since there are 2000 HTML WYSIWYG editors out there.
- narwhal – an alternative to node.js
- npm – a package manager for node.js, similar to gem for ruby
- WebSockets – a new HTML5 feature that allows for better persistent server-client connections. You’ll find some demos around of multiplayer games and web pages that use websockets, and on github there are node.js websocket server examples. You need the Chrome browser or Safari or Firefox 4 beta.
- ExpressJS – server-side web application framework
- Spritely – nice sprite engine
- gameQuery – 2D engine
- Akihabara – for creating arcade-like games
- Copperlicht – WebGL 3D game engine
- GLGE – another WebGL game engine
- SceneJS – WebGL scenegraph library, see also the interactive, editable demos
- C3DL – Canvas 3D library
- Dextrose Aves Engine – commercial game engine in development, see this video talk
- tinygames – see their work (building off of karma edu and so forth) to create educational html5-based games (math only apparently).
- Physics Engines & Demos
Browser-based Development Environments
- See my earlier post Browser-Based IDEs, but also these:
- http://sketchpad.cc/ along with hascanvas and others work with processingjs for creating animations
- I mentioned it in the previous post, but the bespin editor continues to evolve and improve. It now supports code completion using jsctags.
- GUI Designers – really nothing out there that is finished and free and open source (see Ext Designer for a commercial option), other than of course free WYSIWYG HTML designers, but see these rough demos:
So it’s not so surprising that there are now even some programming environments starting to emerge that you can run and develop from inside your browser. These are websites where you can edit code, run your application, and sometimes even instantly share your application for others to try. This is much more convenient than the typical process of downloading and installing a huge IDE such as Eclipse or Netbeans, and then compiling and packaging and distributing/hosting an application on your own. Browser-based environments are also nice for students at schools, because they typically are not given permission to download and install software on their own, but can run these tools in the browser, as long as they aren’t blocked by IT. I imagine it won’t be too long before we start seeing kid-friendly browser-based IDEs start to appear.
UPDATED LIST (December, 2011)
- HTML5 animation design tools (with a timeline like the flash designer)
- https://www.mugeda.com/ – recently announced
- http://mozillapopcorn.org/ - supplement videos with events on a timeline
- (desktop app, preview version is free) http://labs.adobe.com/technologies/edge/
- (desktop app, $199) http://www.sencha.com/products/animator
- (mac only app, $30) http://tumultco.com/hype/
- visual designers for websites
- http://eenox.net/home - design mobile-friendly websites
- http://www.drupalgardens.com/ - lets you visually edit all the css/style properties
- http://maqetta.org/ - visual studio-like IDE for dojo user interfaces
- programming / IDEs
- http://c9.io/ - Cloud 9
- html5 game programming
- http://www.playmycode.com/ (uses a language based on Ruby called Quby)
- http://pixieengine.com/ (coffeescript)
- (desktop app) http://www.scirra.com/construct2
- mobile app creation
- web page development – edit HTML/CSS/JS files on a server
- I switched from Kodingen to DropBox for sharing web pages (including saved Adobe Edge projects): http://www.dropbox.com/help/16
- animation programming – processingjs.org
- see also the jswiki: https://github.com/bebraw/jswiki
- announced but not available yet
Here are some browser-based coding environments I know about. All are free and most are open source.
- [Some updates in July 2011, over a year since this was originally posted:]
- Kodingen – edit php/html/css/etc. files on the server via the browser (I used this in my web development class)
- Maqetta – visual HTML5 editor, builds on the dojo toolkit
- SketchPad.cc – etherpad + processing.js (I used this in my html5 class)
- browser-based game development sites: Pixie (coffeescript), Play My Code (uses a ruby-like language called quby)
- Bespin – [replaced by Ace editor, used in cloud9 IDE listed above] an editor from Mozilla that you can use either on their site, or embed into your own site.
- CodeRun – Develop ASP.NET, php and Ajax applications in your browser. Not open source.
- App Inventor for Android – This is unique in that it has a visual designer, along with a visual blocks-based language for creating applications (similar to that in Lego Mindstorms and Scratch). From this description it does appear to be web-based, although you use it to develop applications that run on an Android cell phone, not the browser. Here’s a course that used the tool last fall, along with the beginnings of a textbook. Unfortunately, it’s still only in private testing and not available to the public. I’ve signed up to use it with students in my multimedia development course this fall. If that works out I’ll post all the resources and videos online, just like all my courses.
- There are also many programming command line interpreters (repls – read-eval-print loop) that work in the browser, such as Lord of the REPLs (lotrepls), which lets you try many different languages out.
- Some other browser-based IDEs: Ares (Palm WebOS development), Zembly (discontinued when Sun was bought out by Oracle), Yahoo Application Platform and Yahoo Pipes, Glowday Google widget designer…
So, what is the platform of choice for folks who want to create interactive graphical educational software (see for example all the stuff at PHET and NLVM). Currently, there are two primary options: Flash and Java Applets. I’m not covering in this post web applications, which can still use just about anything you want: PHP, Java, Ruby, Python, .NET, etc., or business/office/administrative software which can either be web apps or desktop apps coded in C++/Java/.NET/Python, etc., or 3D desktop games, usually coded in C++/Java/.NET/Python. I’m centered on interactive, graphical educational software like you see all over the web now.
The PHET project, for example, uses both java and flash. These haven’t always been the main options, however, and I suspect it will change again in the near future. Here’s a short history of some of the development tools I’ve leaned on for educational software development over the past 15 years:
- early 90s – hypercard / supercard, Perl/CGI for web apps
- early 00s – by this time, commercial options no longer cut it for me, too many bugs, ignored feature requests, too expensive, vendor lock-in – free and open source is king: python, java (later open sourced), C#/vb.net (esp. the Mono open source clone). Unfortunately there is no alternative to the commercial, proprietary flash, which becomes king of RIAs (rich internet applications) instead of java applets.
- early and late 10s? – This is the question of this post. Mobile platforms can no longer be ignored, and that means no java. Actually android is essentially java (dalvik), but standard java applets do not work. Flash is only just now being ported to work on android, and it may become available for the iphone as well, although Apple is hostile to it (as well as java). Since google doesn’t support browser applets (even android applets), and Apple wants total control of their platform, Flash is only increasing its dominance and importance.
Since Oracle bought out Sun, and there is no support nor any planned support for java on the android and iphone platforms, it appears the only open source alternative for the future of RIA apps may be HTML5. But that cannot be used for creating the kind of highly interactive graphical educational software that you can create in java and flash. For this to work in HTML5, it would require WebGL, a 3D (OpenGL ES) canvas for HTML5. WebGL still does not work out of the box on any mobile platform, but it has been or is being ported to work on WebKit (the browser engine for Palm’s WebOS and the iphone web browser) as well as android. Currently, to run WebGL you need to grab a nightly build of Firefox, WebKit, or Google Chrome (I’m trying out the last one). Official builds will have likely have built-in support for WebGL come November 2010 or so perhaps (Firefox 4). Here are some more resources on WebGL:
- Learning WebGL – blog
- Vladimir Vukićević, blog of the main developer for the canvas control, and porting it to android. He did similar work on the mono platform earlier.
- Planet WebGL – aggregator
- Blender to WebGL exporter
- Copperlicht – sample 3D engine for WebGL
- WebGL Wiki – User Contributions (other libraries)
Just a run down on some android apps in case I lose my phone and have to re-install. These are all free. I can’t find many educational apps unfortunately:
- NYTimes – more up to date
- USA Today – more graphical
- 1Cast News – video news stories
- Sports Tap
Social Networking Tools
- Twidroid – there are too many twitter apps out there
- Facebook – comes with Android 2.0 on the droid
- WpToGo – wordpress client
- NewsRob – google reader client
- Barcode Scanner – to read QR codes
- DockRunner – for the droid if you don’t have a multimedia dock – activates the multimedia interface
- Advanced Task Killer – android apps don’t have a quit/exit button
- Astro – file manager – read your SD card, etc. – very nice
- Digital Clock – smaller clock widget
- Gmote – run the gmote server app on your computer and you can remotely control it
- Locale – change android settings based on location or other triggers
- The Weather Channel – there are some other weather apps & widgets too
- Teradesk – store files on the cloud – DropBox app in the works too
- AK Notepad – there are numerous sticky note / notepad apps
- Astrid – there are also numerous todo list apps
- Documents To Go – read word, ppt etc.
- Evernote has a web-accessible site, too, with a forthcoming android client app
- Voice Recorder – app & widget for quick voice notes
- Voice – essential – use for voicemail, forwarding calls, etc.
- Listen – podcasts
- Maps – navigation in android 2.0
- Market – of course to find other software
- Talk – built-in for instant messaging
- Aldiko – incredibly nice e-book reader application
- Google Sky Map – incredible app, point it at sky and it shows the stars & planets
- Labyrinth Lite – tilt game to get ball through maze
- AirAttack – click on planes and helicopters flying overhead to shoot at them
- Air Hockey Demo
- Craigs Race – tilt to race a little car on a track
- Speed Forge 3D – very nice 3D tilt racing game, see this video
- PapiJump – tilt game to bounce a ball upwards
- Robo Defense
- Bubbles – blow on microphone to create bubbles
Music / Video / Photos
- Music – built-in app, plays videos too
- Youtube – built-in app, plays youtube videos embedded on webpages too.
- Qik, Ustream – broadcast live video, and record it
- TV.com – app for watching CBS and related videos. NBC has a mobile-accessible website: m.nbc.com
- Pandora – listen to music online
- Photoshop has an android app now, see also PicSay and others
- Picasa photo album support is built-in to android now
- ConnectBot – ssh
- Amazon, Amazon MP3
- Bubble – bubble level app
I’m also keeping an eye out for note taking / handwriting recognition apps for a research project:
Well this has little to do with education, other than Android is the only smart phone platform that you can quickly and for no cost develop educational software for I suppose. The other platforms such as iPhone and Palm Pre are either not free and open source, and/or take forever to make your app available for others to use and cost money to develop for. There is still the J2ME platform too, but I’m limiting this to newer generation smart phones.
There are now (in the U.S.) 4 Android phones to choose from. Well, 2 of them are not yet released but will be soon. All are basically identical under the hood: processor speed, touchscreen, camera, bluetooth, wireless, GPS, etc. This engadget post compares the technical features of the different Android phones, but I wanted to mention a bit more about these 4 options:
- T-Mobile G1 (or “Dream”, $97 at walmart/amazon + 2 year contract) – This has been out a year now. Features slide out keyboard, 3.2MP camera. No headphone jack, only USB. A little bigger than the newer Android phones below.
- T-Mobile MyTouch (or “Magic”, $99 with Oprah coupon code + 2 year contract) – Released last month, this is the second generation of the G1 but still basically the same software-wise. It is smaller partly because it has no hardware slide out keyboard. Slightly bigger battery. Also no headphone jack, only USB.
- Sprint Hero ($279 at bestbuy or sprint + 2 year contract – $100 rebate) – To be released October 11th (see announcement). Has more tailored and polished software (HTC Sense). Has a 5MP camera, but no keyboard. Supports Sprint features like Sprint TV. Sprint supposedly has 20x the high speed 3G network coverage as T-Mobile but may add roaming charges.
- T-Mobile Cliq (?? + 2 year contract) – Due out later this fall. Smaller form factor, has a slide out keyboard and enhanced MOTOBLUR software tailored for folks who use social networking sites like Facebook and Twitter. See this demo.
For all the phones you have to get a voice plan + a data plan. Starts around $65/month at T-mobile ($85 for family plan) and $69/month or so at Sprint ($129 for family plan).
Some popular third party Android software includes ShopSavvy (scan barcodes to compare prices), Pandora (listen to music online like the Pandora website), and Repligo (read pdfs).