ContentNotEditable: What the “death” of the mouse and keyboard means for content creators
First let me start with some disclaimers to try to make sure this post is not misinterpreted: I am not arguing that the mouse and keyboard are really dead or that the lack of a mouse on tablets is a bad thing. I am not arguing that the ipad or similar devices are awful for education or content creation. I am just thinking about how to make them even better in these areas, and conceptual and technological roadblocks in the way. Some of the criticisms of the ipad as a content consumption device (here, here), have been addressed with the ipad 2 and other new android tablets with their inclusion of cameras and input ports, and some tablets are even coming out with a stylus, like the HTC Flyer.
But imagine any creative person – creating a 3d character for a game, drawing a picture, composing a music score, creating a graphically rich document or presentation, etc. They likely have something in their hand, or their hands are busy doing something. When that involves interacting with a computer they are likely clicking the mouse to drag something around or edit text, for example, or using a stylus on a digitizer surface (like a wacom) for drawing, or typing away on a keyboard.
These input devices are all essentially gone on new tablets and smartphones. You can still type (slower) on virtual keyboards, and you can click like a mouse with your finger (tap).
An example of the impact of this is rich text editing, like with a word or openoffice or google docs document. Many browser-based wysiwyg editing tools, which are used virtually everywhere, such as in moodle (which uses the TinyMCE editor) or drupal, no longer work when you access them from an ipad or iphone or android device (or other mobile platforms like blackberry or palm webos). Even the newest “HTML5″ editors, such as Aloha Editor, pop up an error message if you try to access them from a mobile device. Other browser-based editing and drawing tools also no longer work on these new platforms, or you have to draw with your fingers. Most of Google’s and others’ tools like Google Presentation do not work on mobile platforms. Really, just imagine most any software people use on a desktop to create stuff – like office, or the flash ide, or gimp/photoshop, blender 3d, etc. Even when programming, which really is just typing in plain text, we usually prefer to use IDEs that popup suggestions and corrections to help us out. For many of our desktop apps its hard to even imagine them working on a tablet or phone.
The rich text editing tools in browsers like TinyMCE or CKEditor primarily rely on the contenteditable HTML attribute to support editing. Add that attribute to an HTML element, and the contents of that element become editable inside the web browser. It works in all browsers, including old Internet Explorer versions. It doesn’t really work well or even at all on mobile browsers though (see here, here, here, here).
Newer versions of android, webkit, and mobile firefox have been slowly improving their support for contenteditable, and maybe they will eventually “fix” the issue, but I’m not sure that this will be fixed through engineering alone. Some code editors like codemirror 2 and the ace editor are trying out workarounds like using a hidden textfield that captures key presses. Codemirror 2 works on an ipad somewhat, the ace editor does not. It remains to be seen if a similar trick might work for a rich text editor (it’s tricky enough just to do it for plain text). And like I said, the HTC Flyer and other tablets (esp. those being designed for medical and other professionals) are starting to include a stylus, and it remains to be seen if that will catch on (it didn’t before with older tablets). Others are coming out with dual screen tablets, where the second touchscreen can work like a touchpad on a laptop or nintendo ds, but that also may not catch on.
Another more general alternative strategy to this issue of tablets having no mouse or other input devices other than the touchscreen and the camera (which can be used for gestural or other input), might be to conceptually rethink how to support multimedia creation on these mobile platforms. Perhaps we should drop the notion of “documents” or “pages”. After all, you don’t think of a flash widget as a page or document. You don’t think of a game as a set of pages or documents. And Apple and other developers have already created apps for some specialized types of content creation and creativity, such as musical instrument simulators and so forth.
So, this may be a pre-paradigmatic moment where we’ll see what catches on: will we try to perfectly “emulate” the mouse and stylus and its supported interactions via other means such as gestures, or will new and unique types of interactions continue to catch on (like multitouch stuff). Probably a combination of both, but so far the camera is hardly being used at all for input, other than recording videos or taking pictures. And some may dismiss the idea of a stylus ever catching on again, but Apple has occasionally made “mistakes” before (the first mac didn’t have a floppy drive, for example), and others have been successful in incorporating a stylus, like the Nintendo DS, which my little boy continually loses
Some Exciting New HTML5/Javascript Projects
This an update of sorts on some things happening in the HTML5/Javascript world. First, here are some previous posts on this topic, to catch you up to speed:
- What is the Platform of the Future for Developing Interactive Graphical Educational Software? – I argue for HTML5/Javascript, since Java and Flash do not run on (all) mobile/tablet platforms
- A Snapshot of the HTML5/Javascript Universe – a listing of various open source projects, libraries, and tools available to use
- Browser-Based IDEs – this is the future, I believe – doing coding in the browser. Some updates are discussed below (cloud9 IDE, and a web-based version of eclipse)
Some Exciting New HTML5/Javascript and Related Projects
Here are some interesting new software projects and libraries and tools I’ve run across since my previous HTML5 post:
- Mobl
- http://www.mobl-lang.org/ and https://github.com/mobl
- Mobl is a new language that compiles down to javascript. It has a great number of interesting features, including static typing, IDE integration (Eclipse), reactive programming (binding), data persistence (the creator of Mobl Zef Hemel also created persistence.js), and it automatically converts synchronous to asynchronous code. This project is still under active development, but a great deal has already been implemented.
- Other Javascript-based languages and compilers
- Here is a long list: https://github.com/jashkenas/coffee-script/wiki/List-of-languages-that-compile-to-JS Coffeescript is the most popular, but it sticks to the boundaries of what javascript handles (so no asynchronous macros or static typing or other big changes, for example)
- Modkit
- http://www.modk.it/software
- Modkit is basically a version of the Scratch graphical programming environment, but it works in the browser. Not yet released, but coming soon.
- WireIt
- http://neyric.github.com/wireit/ and https://github.com/neyric/wireit
- WireIt is a YUI-based library for creating graphical interfaces or libraries that are like draggable nodes or concept maps, except the nodes aren’t merely bubbles or text, they can be editors or controls or whatever.
- HTML5 Canvas libraries
- This is a rapidly evolving area as well. Any graphical, educational animation or simulation is going to revolve around the (relatively) new <canvas> tag in HTML5. There are several new canvas libraries out there, but there isn’t yet one clearly superior option that has the best of all the features of these libraries. Another factor is that the javascript universe is split between jquery and non-jquery libraries (jquery mobile is emerging as a top option for the non-canvas parts of an application interface). Some of these rely on jquery and some do not:
- CAAT http://labs.hyperandroid.com/animation and https://github.com/hyperandroid/CAAT/
- Unveil.js https://github.com/michael/unveil – Has support for not using up the processor when it is not needed
- easeljs http://easeljs.com/ – Created to be like flash with display lists and so forth, by flash developers. Works with jquery I believe.
- doodle.js http://lamberta.posterous.com/doodle-js and https://github.com/biilly/doodle-js
- artisan.js http://artisanjs.com/ and https://github.com/davidbrooks/Artisan
- sprite.js https://github.com/batiste/sprite.js
- https://github.com/dkln/canvas_library
- gury https://github.com/rsandor/gury – Works with jquery
- Javascript diagramming/flowchart libraries
- New game libraries
- Gesture recognition libraries
- Browser-based IDEs
- The Cloud9 IDE is under heavy development, and they’ve now opened up beta access to the hosted version of their javascript (node.js) based development environment. Their Ace editor component is available now as a separate project, as well, which has combined with Mozilla’s skywriter editor project.
- In June or so, look for an initial release of a web-based version of the Eclipse IDE, codenamed Orion
- 3D Multiuser World
- It has long been a holy grail of the edtech world to have a multi-user virtual world for students and teachers to use. There already is Second Life (and it’s mono-based open source clone OpenSim) and Open Wonderland (java-based), but now with WebGL, it’s possible to create such a thing that works in the browser, and Katalabs has done so, releasing a prototype of their open source virtual world software called Kataspace. Requires a recent version of firefox or chrome – see the Learning WebGL site for more info on WebGL.
- BASIC in the browser
- Some various compilers/editors for basic or visual-basic like languages that work in the browser:
- Pex: http://www.pexforfun.com/Page.aspx#learn/
- qb.js: QBASIC in the browser http://stevehanov.ca/blog/index.php?id=92
- Smalltalk/Squeak in the browser
- Proper
- http://quasipartikel.at/proper/
- Proper is a semantic text editor that works in the browser. See also other related libraries like substance, data, and unveil: https://github.com/michael
- Real-time Collaboration/Editing
- IBM just announced the Open Cooperative Web Framework, which uses Dojo and other libraries for real-time collaboration in the browser. Targeted for things like collaborating during a conference.
- Tutto is a real-time collaborative javascript shell, using websockets: http://tobyho.com/Tutti_-_Multiple_Browser_Interactive_Javascript_Shell
- Realie is a web-based real-time collaborative editor, similar to etherpad. Unlike etherpad, it is coded with node.js and websockets: http://laktek.com/2010/05/25/real-time-collaborative-editing-with-websockets-node-js-redis/
- Here are some other browser-based collaborative real-time editors: http://en.wikipedia.org/wiki/Collaborative_real-time_editor#Browser-based
- Java Projects
- These are not javascript at all but I thought I’d share them:
- Visage http://code.google.com/p/visage/ open source port of JavaFX, which declarative markup for creating user interfaces
- Openxion http://code.google.com/p/openxion/ Openxion is a java-based port of Hypercard (more accurately, hypercard’s language hypertalk). It is a command-line tool, it doesn’t support the graphical stuff (cards, etc.) hypercard did. Interestly, when javascript first came out it was touted as a ‘hypercard for the web’, but really it isn’t even close to what hypercard could do.
- Wireframes and Mockups, Sketches
- There are several new browser-based tools for visually designing interfaces (fake interfaces), some of which are HTML5-based or else Flash: http://speckyboy.com/2010/01/11/10-completely-free-wireframe-and-mockup-applications/
- Some open source ones include: http://www.k-sketch.org/ http://www.mdaines.com/plumb/ http://dub.washington.edu:2007/projects/sketchwizard/
- OpenSocial/widget development tools
- Socialtext has an HTML5 widget builder: http://www.readwriteweb.com/enterprise/2010/12/socialtext-introduces-html5-ba.php
- Exo is a competitor to Socialtext, and they also have a web-based IDE: http://exoplatform.com/company/en/resource-viewer/Video-Demo/building-web-app-with-exo-platform-3-ide
- Others
- UJam is a site that lets you put together or record your own music from inside the browser: http://www.ujam.com/
- and of course there are several other html5-based ‘web 2.0′ tools for creating other stuff in the browser, like Aviary for image editing: http://www.aviary.com/
Recent Course & Workshop Resources: HTML5, Drupal, Distance Education
Here are some web sites for courses I’m currently teaching or recently taught:
- Distance Education Tools - http://detools.wikispaces.com/
- This site now has dozens of links of various tools and technologies you can use with online and blended learning
- Multimedia Development with HTML5 - http://html5.wikispaces.com/
- As I posted earlier, HTML5 is emerging as an alternative to flash and java
- Instructional Design Projects – http://holton.usu.edu/moodle
- We learn how to use a learning management system (moodle) and about “post-modern” instructional design techniques, such as constructivism, problem-based learning, simulations, games, cases…
- Internet Development: http://internetdev.usu.edu/outline
- I’ll be teaching web development with the Drupal content management system again next summer. I hope to improve the videos.
At the AECT conference last week I gave a workshop on using Drupal to create web applications – here’s a Drupal Cheat Sheet I made for it. And here are slides for a talk on embodied cognition and instructional design.
A Snapshot of the HTML5/Javascript Universe
This is a follow-up to “What’s the Platform of the Future for Developing Interactive Graphical Educational Software?“, which I argued is HTML5, especially since java applets and flash don’t work or aren’t well supported on mobile and emerging tablet platforms (like the ipad and android tablets). I’m lumping many things under the “HTML5″ moniker, including the HTML5 Canvas element, WebGL for 3D, and various Javascript and CSS frameworks and libraries, even though most of the latter weren’t designed for HTML5 in particular.
Here’s a taste of the current tools and frameworks out there for developers interested in learning more about this platform. Probably the first thing you have to decide though, is are you more interested in running your HTML5 app on mobile phones and tablets (iPhone and iPad’s iOS, Android, and to some extent Blackberry and Palm), or in a regular desktop browser (Firefox, Chrome, Safari, IE…), or both? Some tools for mobile web app development are listed below.
Keeping Up with the HTML5/Javascript/CSS Universe
First, here are some places to keep up with this rapidly evolving field:
- http://twitter.com/edtechdev/html5 – A list of folks and organizations that develop javascript & html5 related things
- Learning WebGL – for keeping up with 3D developments
- Some discussion groups: http://groups.google.com/group/iphonewebdev , sitepoint forum, codingforums, etc.
- The Changelog – They have a regular podcast which often features interviews with developers of HTML tools such as some of the ones listed below (Sencha Touch, Coffeescript, etc.)
- There are several other javascript and css and html5 news sites and blogs out there, too, such as Badass Javascript.
General Javascript Application Frameworks & UI Toolkits
- JQuery – most popular and almost universal javascript framework. It lets you ‘select’ elements in the DOM to operate on, in a very functional way. See numerous jquery tutorials, books and plugins and jquery.ui. Has a little mobile support (touch/swipe).
- YUI – very java-like user interface toolkit from Yahoo, very complete and accessible. They have started adding mobile support (touch/swipe/etc).
- See this comparison of javascript frameworks for others such as Sproutcore, Dojo, qooxdoo, mootools, etc. Some other frameworks listed on that page actually involve coding in Java (or another language) which is converted to javascript, such as GWT and pyjamas.
- JS Optimizers – to compress/obfuscate and combine into one file your js code see Google Closure Compiler, YUI Compressor, JSMIN
- Mobile application frameworks – these frameworks are specifically for creating mobile applications (iphone/ipad/android) using javascript/html5, although usually that doesn’t include the canvas yet, and also unfortunately most of their demos work ONLY on mobile platforms, not on the desktop. The main problem is that on the desktop with a mouse you can click and drag and drop, whereas on mobile/tablets with no mouse you touch and swipe and so forth instead. One decision you also need to make is, do you want to access native elements of the mobile platform (like camera, tilt sensor, etc.). If so, see the first two options (PhoneGap and Appcelerator). The latter options often work with PhoneGap, too.
- PhoneGap – has an open source permissive license. They support Blackberry and Palm and Symbian and so forth, as well as iOS and Android. They create a native webview wrapper for your HTML5 app, so that you can access native things such as the camera or sensors.
- Titanium from Appcelerator – also has an open source permissive license, for fast, native mobile apps developed with javascript/html5 that can access native sensors and so forth, too. See their kitchen sink demo and other demos.
- Sencha Touch – built off of ExtJS and JQTouch – any app you develop with it either needs to be GPL or else you have to pay for a commercial license from them. Can work with PhoneGap, they have some nice demos, esp. for the iPad.
- Ansca Mobile – Corona SDK, better for game development, commercial license.
- appMobi – commercial, too, I believe
- Rhomobile – you actually develop in Ruby and it converts it to a javascript/html5 mobile app
HTML5 Canvas & Javascript Drawing / SVG Libraries
There are hundreds of html5 canvas demos out there if you search around. Again, unfortunately many canvas demos do not work as well on mobile or tablet platforms, or lack interaction support (drag or swipe or whatever). The harmony demo listed below is one exception. Android 2.2 will incorporate a much faster v8 javascript engine from google, and future versions of the webkit browser used by most mobile/tablet devices will hopefully add support for 3D with webgl, too. See the WebGL / game development stuff in a later section below.
- Canvas tutorials
- ProcessingJS – nice drawing library and development environment for creating animations and visualizations. This is a port by John Resig, inventor of JQuery, and others of the original java-based processing library to javascript. There are numerous sites that let you code your processingjs program from within your browser (see “browser-based IDEs” below), such as hascanvas and jsdo.it.
- canto.js – improved canvas api, very jquery-like, includes some turtle graphics (logo-like) commands. See also this turtle graphics in javascript page.
- ExCanvas – for making canvas work in internet explorer (IE 9 will support it I believe)
- Raphael – Javascript SVG, vector graphics, library – very nice demos
- RGraph – graphing library
- flot – javascript plotting library
- bluff – graphing library
- Javascript InfoVis Toolkit
- three.js – canvas-based 3d engine (not using webgl as the ones below)
- Painting/Sketching demos
- Harmony – by mrdoob, who also created the above three.js. This is a very nice demo which also works on mobile/tablet platforms, with touch support.
- Canvas Painter
- sketchpad
Alternate Languages/Parsers/Class systems for Javascript/CSS/HTML
As I mentioned in the previous post, one potential downside of the HTML5 platform (at least from a beginner’s perspective), is that you are stuck with the javascript language (as well as HTML and CSS). Well, that’s not always the case now. There are alternate languages that compile to javascript (like coffeescript) or css (like Sass and Compass) or even HTML (like HAML). Javascript, in a sense, is the new C. Many of the types of tools we used to mainly see in Java-land are also now available in javascript, such as parser generators.
- Coffeescript – alternate language that compiles to javascript. It is still mostly like javascript, but adds some things that make your code much more concise.
- Sass and Compass – alternative to CSS – adds some smarter features to CSS like variables and so forth
- HAML – alternate to HTML, not as popular perhaps, or as needed as the above two, since there are 2000 HTML WYSIWYG editors out there.
- Javascript parser generators – if you want to create your own DSL or language:
- Actionscript to javascript – see Jangaroo and Gordon
- If you want a more ruby or java-like class system (inheritance, traits, mixins, design by contract,etc.) to use rather than javascript’s built-in prototype object system, see:
Server-Side Javascript
Javascript is no longer a language that only runs in the browser. It has become a first class language for the server / desktop, too. Just run “node myscript.js” for example like you would any other kind of script on a server or linux box. Coffeescript works with node.js on the server or desktop, too.
- node.js – Uses Google’s fast V8 javascript runtime.
- narwhal – an alternative to node.js
- CommonJS – a standard library for javascript, mainly meant for the server-side at this point. node.js and narwhal implement commonjs.
- npm – a package manager for node.js, similar to gem for ruby
- I’m still waiting to see a drupal-like content management system built all in javascript.
- Jake – a build tool for javascript (like make, rake, etc.)
- WebSockets – a new HTML5 feature that allows for better persistent server-client connections. You’ll find some demos around of multiplayer games and web pages that use websockets, and on github there are node.js websocket server examples. You need the Chrome browser or Safari or Firefox 4 beta.
- ExpressJS – server-side web application framework
Javascript Game Development, 3D & Physics Engines
Further evidence that javascript is the real deal, you can do 3D and interactive games with it. To run the WebGL stuff you’ll need a recent build of the Chrome (or Chromium) browser or Firefox, see the Learning WebGL FAQ section.
- Spritely – nice sprite engine
- gameQuery – 2D engine
- Akihabara – for creating arcade-like games
- jsGameSoup
- Copperlicht – WebGL 3D game engine
- GLGE – another WebGL game engine
- SceneJS – WebGL scenegraph library, see also the interactive, editable demos
- C3DL – Canvas 3D library
- Dextrose Aves Engine – commercial game engine in development, see this video talk
- tinygames – see their work (building off of karma edu and so forth) to create educational html5-based games (math only apparently).
- Physics Engines & Demos
Browser-based Development Environments
- See my earlier post Browser-Based IDEs, but also these:
- jsdo.it – Has built-in support for javascript libraries such as processing, jquery, etc., and you can share/fork/rate code. jsFiddle is a similar tool.
- http://sketchpad.cc/ along with hascanvas and others work with processingjs for creating animations
- I mentioned it in the previous post, but the bespin editor continues to evolve and improve. It now supports code completion using jsctags.
- GUI Designers – really nothing out there that is finished and free and open source (see Ext Designer for a commercial option), other than of course free WYSIWYG HTML designers, but see these rough demos:
Browser-Based IDEs (programming environments)
As I wrote about earlier, HTML5 (Javascript, Canvas, WebGL, etc.) is allowing people to create rich, interactive applications that run in your browser and don’t require flash or java. Cloud computing (such as Google App Engine and others) and AJAX are also allowing us to run things in our browser that before were typically run on the desktop because they require quickly saving and retrieving a bunch of info. Google docs, gmail, etc.
So it’s not so surprising that there are now even some programming environments starting to emerge that you can run and develop from inside your browser. These are websites where you can edit code, run your application, and sometimes even instantly share your application for others to try. This is much more convenient than the typical process of downloading and installing a huge IDE such as Eclipse or Netbeans, and then compiling and packaging and distributing/hosting an application on your own. Browser-based environments are also nice for students at schools, because they typically are not given permission to download and install software on their own, but can run these tools in the browser, as long as they aren’t blocked by IT. I imagine it won’t be too long before we start seeing kid-friendly browser-based IDEs start to appear.
UPDATED LIST (December, 2011)
- HTML5 animation design tools (with a timeline like the flash designer)
- https://www.mugeda.com/ – recently announced
- http://editroom.splatcollision.com/
- http://mozillapopcorn.org/ - supplement videos with events on a timeline
- (desktop app, preview version is free) http://labs.adobe.com/technologies/edge/
- (desktop app, $199) http://www.sencha.com/products/animator
- (mac only app, $30) http://tumultco.com/hype/
- visual designers for websites
- http://eenox.net/home - design mobile-friendly websites
- http://www.drupalgardens.com/ - lets you visually edit all the css/style properties
- http://maqetta.org/ - visual studio-like IDE for dojo user interfaces
- programming / IDEs
- http://c9.io/ - Cloud 9
- http://eclipse.org/orion/
- http://www.akshell.com/ide/
- http://www.erbix.com/js/
- http://www.pageforest.com/
- html5 game programming
- http://www.playmycode.com/ (uses a language based on Ruby called Quby)
- http://pixieengine.com/ (coffeescript)
- (desktop app) http://www.scirra.com/construct2
- mobile app creation
- http://www.appsbar.com/
- other mobile app builder/services: http://bit.ly/sdHaUX
- web page development – edit HTML/CSS/JS files on a server
- https://kodingen.com/
- http://www.coderun.com/
- I switched from Kodingen to DropBox for sharing web pages (including saved Adobe Edge projects): http://www.dropbox.com/help/16
- html/css/javascript editors – quickly run & share snippets
- animation programming – processingjs.org
- alternative languages for javascript – some of which have online editors/compilers:
- https://github.com/jashkenas/coffee-script/wiki/List-of-languages-that-compile-to-JS
- see also the jswiki: https://github.com/bebraw/jswiki
- announced but not available yet
- http://www.animatable.com/
- http://www.thepixeljet.com/preview/
- http://html5.yoyogames.com/
- http://www.automatoon.com/ – will be re-released at teachtoon.com
OLD NOTES:
Here are some browser-based coding environments I know about. All are free and most are open source.
- [Some updates in July 2011, over a year since this was originally posted:]
- Cloud9 IDE – create server-side node.js javascript apps in this browser-based development environment
- Erbix - another Javascript App editor
- Akshell – and another Javascript application editor
- Kodingen – edit php/html/css/etc. files on the server via the browser (I used this in my web development class)
- Maqetta – visual HTML5 editor, builds on the dojo toolkit
- SketchPad.cc – etherpad + processing.js (I used this in my html5 class)
- Zeon.js – still early in development, not open source – analyzes javascript source code for issues
- other javascript/html/css online editors: jsFiddle, jsdo.it, jsbin
- browser-based game development sites: Pixie (coffeescript), Play My Code (uses a ruby-like language called quby)
- Alternative languages that compile to javascript - a huge list, some of which let you edit and compile scripts in the browser.
- HasCanvas – an IDE for Processing.js. Processing is a very popular java-based development environment, with a free IDE you can download. Processing.js is a port of that to javascript. With either one, you can create animations and artwork and so forth. HasCanvas has a weird interface – hover over the arrows on the left side to show the source code for any animation, and use your mouse wheel to scroll. Click Browse to see other scripts.
- Bespin – [replaced by Ace editor, used in cloud9 IDE listed above] an editor from Mozilla that you can use either on their site, or embed into your own site.
- CodeRun – Develop ASP.NET, php and Ajax applications in your browser. Not open source.
- App Inventor for Android – This is unique in that it has a visual designer, along with a visual blocks-based language for creating applications (similar to that in Lego Mindstorms and Scratch). From this description it does appear to be web-based, although you use it to develop applications that run on an Android cell phone, not the browser. Here’s a course that used the tool last fall, along with the beginnings of a textbook. Unfortunately, it’s still only in private testing and not available to the public. I’ve signed up to use it with students in my multimedia development course this fall. If that works out I’ll post all the resources and videos online, just like all my courses.
- There are also many programming command line interpreters (repls – read-eval-print loop) that work in the browser, such as Lord of the REPLs (lotrepls), which lets you try many different languages out.
- Some other browser-based IDEs: Ares (Palm WebOS development), Zembly (discontinued when Sun was bought out by Oracle), Yahoo Application Platform and Yahoo Pipes, Glowday Google widget designer…
Need Much Much More “D” in Education R & D
I already blogged about this matter 3 years ago in a post entitled “The State of Educational Research & Development.” But a few recent things made me think of it again:
- @newsweek tweeted for us to tell them our thoughts on the American education system in 6 words or less. My first thought was that “education needs more r&d”, because as my previous post mentioned – medicine and engineering and related areas spend 5-10% on research, whereas in education that percentage is closer to 0.01%. And most of that miniscule amount is spent on basic research, not development. NSF doesn’t even fund much K-12 or higher education curriculum or software development anymore.
- The U.S. department of education recently released its National Educational Technology Plan for 2010, authored by many researchers with whom I am familiar. I’ve only scanned it so far, but I haven’t seen much emphasis on development, just research. The only development ideas I’ve seen so far are very top-driven solutions, the “Grand Challenges” described in the end section on R&D: for example a huge tutoring system, a system for delivering assessments, a school data sharing and mining system, etc. I’m not seeing any bottom-driven or domain-specific ideas or more specific solutions. I posted a comment on that page similar to this post.
- Tony Bates blogged about “the state of e-learning in 2009” and noted:
My biggest disappointment this year…has been with open educational resources…what are we getting? Digitally recorded 50 minute classroom lectures and digital textbooks. What we are not getting are materials designed from scratch for multiple use…And there is still so little of it. What I would like to see are many thousands of short modules
- Also as I wrote in another earlier post on “50 examples of the need to improve college teaching,” software is key. The National Center for Academic Transformation (NCAT) has helped people redesign their college courses to be much more effective and efficient and cost less. The key to that is the use of interactive software: “successful course redesign that improves student learning while reducing instructional costs is heavily dependent upon high-quality, interactive learning materials” (ref). That may work well for common, large enrollment courses for which there is already a bunch of software available, but what about the rest of the courses? What about all of K-12, too?
- I recently wrote a chapter titled “Toward a Nation of Educoders” about how if we could make it easier for students and teachers to develop interactive software (such as animations and games and interactive websites), perhaps this would help alleviate this problem, make it less formidable and daunting, financially and timewise. This is related to the “computational thinking” (pdf) and “computational literacy” push seen in computer science. We should look at programming as the new 4th “R”, a new literacy that students and teachers need in today’s world. A couple years ago I blogged about this and an article by Marc Prensky: “Programming: The New Literacy.”
What’s the Platform of the Future for Developing Interactive Graphical Educational Software?
So, what is the platform of choice for folks who want to create interactive graphical educational software (see for example all the stuff at PHET and NLVM). Currently, there are two primary options: Flash and Java Applets. I’m not covering in this post web applications, which can still use just about anything you want: PHP, Java, Ruby, Python, .NET, etc., or business/office/administrative software which can either be web apps or desktop apps coded in C++/Java/.NET/Python, etc., or 3D desktop games, usually coded in C++/Java/.NET/Python. I’m centered on interactive, graphical educational software like you see all over the web now.
The PHET project, for example, uses both java and flash. These haven’t always been the main options, however, and I suspect it will change again in the near future. Here’s a short history of some of the development tools I’ve leaned on for educational software development over the past 15 years:
- early 90s – hypercard / supercard, Perl/CGI for web apps
- late 90s – java, visual basic, real basic, PHP emerges for web apps, javascript in the browser
- early 00s – by this time, commercial options no longer cut it for me, too many bugs, ignored feature requests, too expensive, vendor lock-in – free and open source is king: python, java (later open sourced), C#/vb.net (esp. the Mono open source clone). Unfortunately there is no alternative to the commercial, proprietary flash, which becomes king of RIAs (rich internet applications) instead of java applets.
- late 00s – by this time, desktop apps no longer cut it. RIA is king for creating interactive graphical educational apps – java and JVM languages like scala, and still no real alternative to flash (there is javafx, but it hasn’t largely caught on yet, and just about all the demos so far require you to download the app (JNLP) rather than run in the browser, despite support for JNLP applets). The move to RIA means no more Mono/.NET since it doesn’t run in the browser – support for silverlight is weak/non-existent on many browsers/platforms. And no python, although perhaps one day browsers will support it as an alternative language to javascript. I’m not holding my breath on that though. Mono/.NET is emerging again now in 3D virtual world space, however, because it is the basis for Second Life / OpenSim, and its only open source competitor, the java-based Project Wonderland is no longer supported by Oracle, who bought out Sun. The project is continuing outside of Oracle, but it’s future is unclear.
- early and late 10s? – This is the question of this post. Mobile platforms can no longer be ignored, and that means no java. Actually android is essentially java (dalvik), but standard java applets do not work. Flash is only just now being ported to work on android, and it may become available for the iphone as well, although Apple is hostile to it (as well as java). Since google doesn’t support browser applets (even android applets), and Apple wants total control of their platform, Flash is only increasing its dominance and importance.
HTML5
Since Oracle bought out Sun, and there is no support nor any planned support for java on the android and iphone platforms, it appears the only open source alternative for the future of RIA apps may be HTML5. But that cannot be used for creating the kind of highly interactive graphical educational software that you can create in java and flash. For this to work in HTML5, it would require WebGL, a 3D (OpenGL ES) canvas for HTML5. WebGL still does not work out of the box on any mobile platform, but it has been or is being ported to work on WebKit (the browser engine for Palm’s WebOS and the iphone web browser) as well as android. Currently, to run WebGL you need to grab a nightly build of Firefox, WebKit, or Google Chrome (I’m trying out the last one). Official builds will have likely have built-in support for WebGL come November 2010 or so perhaps (Firefox 4). Here are some more resources on WebGL:
- Learning WebGL – blog
- Vladimir Vukićević, blog of the main developer for the canvas control, and porting it to android. He did similar work on the mono platform earlier.
- Planet WebGL – aggregator
- Blender to WebGL exporter
- Copperlicht – sample 3D engine for WebGL
- WebGL Wiki – User Contributions (other libraries)
- O3D is an open source google project (they also support WebGL), that is essentially a 3D canvas with a more javascript-friendly API. It seems to be better for static 3D scenes with less interactivity. It is also NOT supported on android or other mobile platforms.
The main drawback to the HTML5 platform is that javascript is the only language supported. That’s not a problem for me, personally, I’ve been using javascript since when it was called livescript. But my interest also is in programming languages/tools that make it easier for students, teachers, and other non-CS types to develop interactive educational software, as discussed in this chapter (pdf). A workaround for now would be to create a to-javascript compiler for alternate languages, as has already been done for java with the GWT project (which has various WebGL controls in development: WGT, GWT-G3D, GWTGL, GWT-WebGL), but in the long run it would be best if a common runtime were developed for WebKit and Firefox to support other languages than javascript, but then we are just re-inventing java and the JVM. So I don’t dismiss the java platform at all just yet, but it’s definitely not having a good year so far
Misconceptions about Design
Design is the process of going from function to structure. There is some purpose, or goal, or effect on the environment desired (a function), and structures are created or organized to achieve that function. See more about structure-behavior-function models of systems in this post and more about design generally on this What is Design? page from the Design in the Classroom site. There has been much written about the process, method, or steps of design, including various models and design cycles.
There are of course numerous forms and types of design – engineering design, web design, software design, architectural design, instructional design, visual design, interaction design, and so on. Design Studies is a journal that covers design broadly, and there are numerous journals devoted to domain-specific areas and forms of design, such as Instructional Science and Educational Technology, Research & Development (instructional design), Artificial Intelligence for Engineering Design, Analysis & Manufacturing (engineering design), … There’s even a kids’ show called Design Squad that covers design more broadly as well.
What has not been well researched in virtually all areas of design are the misconceptions or preconceptions and conceptual hurdles people have when designing or learning to design (especially beginners and students, but also experienced designers as well, as will be illustrated below). Misconceptions have been well researched in areas such as science education (see this book for example) and history education (see this book), but not in design areas such as engineering, web design, or even instructional design. Why is this important? The book How People Learn highlights 3 key findings that we’ve learned from research on learning and teaching, and #1 is the need to identify and confront students’ initial knowledge and misconceptions they have when learning a new subject or skill:
“Students come to the classroom with preconceptions about how the world works. If their initial understanding is not engaged, they may fail to grasp the new concepts and information that are taught, or they may learn them for purposes of a test but revert to their preconceptions outside the classroom”
“Teachers need to pay attention to the incomplete understandings, the false beliefs, and the naive renditions of concepts that learners bring with them to a given subject.”
So, with all that in mind, below are just a few examples of misconceptions about design in various domain areas that I’ve found. Like I said, there is not much actual research or data out there, just some anecdotal resources – I welcome any comments with other examples of misconceptions about design, and perhaps it is an area you or I or others will explore further in the future.
Software Design – Anti-Patterns
Actually this is one design area in which there is a huge amount of anecdotal evidence for misconceptions. See this list of anti-patterns, for example. Whereas computer science design patterns are common solutions to problems that occur in software design, an anti-pattern is a commonly used yet often ineffective or counter-productive design pattern. Anti-patterns aren’t all really ‘misconceptions’, but just as noted in the How People Learn quotes above, it makes sense to be aware of anti-patterns so that you don’t repeat the same mistakes many other designers have made.
It would be interesting to see how many of these software design anti-patterns may apply to other forms of design as well. I’m not a computer science teacher nor was I a CS student, but I do not get the sense that anti-patterns (nor design patterns, for that matter) are covered or addressed in software engineering courses. Perhaps they should.
Engineering Design
“Engineering design is the process of devising a system, component, or process to meet desired needs. It is a decision-making process (often iterative), in which the basic sciences, mathematics, and the engineering sciences are applied to convert resources optimally to meet these stated needs” (from p. 3 of ABET criteria).
So far I’ve mainly only found one short paper about misconceptions in engineering design. Wendy Newstetter and others surveyed freshmen in an engineering design course (the ACM paper is not publicly accessible), and listed 5 main misconceptions about design:
- Ideation without substance – Students believe that design is coming up with good ideas. And that’s it. They forget about the rest of it – how to realize these ideas and evaluate them.
- Design arrogance – Students forget the constraints of the environment in which the design will reside. They “arrogantly” ignore the constraints of the user.
- Design fixation – (perhaps related to the idea of “functional fixedness” as well) Students tend to focus on the first solution that comes to mind. They stop considering alternatives.
- Extreme design -Students focus only on the very high level (function) or the very low level (structure), without moving between them in a formal manner and considering the giant gulf between the two levels.
- Design serialization – Students belief that design is a serial/linear process, ignoring iterative cycles, revisiting past decisions, and evaluating alternatives.
Instructional Design – ISD
People in the instructional technology/design field will recognize some of the above misconceptions. The first thing I teach in my advanced instructional design course is misconceptions about ADDIE (a popular formal instructional design model). Even the author of the most popular textbook that teaches ADDIE (Walter Dick), stated that students shouldn’t view ADDIE as a linear recipe to be followed (I don’t have the quote with me).
Rieber quotes Michael Streibel (1991, p. 12) about the difference between instructional design models (such as ADDIE) taught in the classroom and instructional design as it is actually practiced:
I first encountered the problematic relationship between plans and situated actions when, after years of trying to follow Gagné’s theory of instructional design, I repeatedly found myself, as an instructional designer, making ad hoc decisions throughout the design and development process. At first, I attributed this discrepancy to my own inexperience as an instructional designer. Later, when I became more experienced, I attributed it to the incompleteness of instructional design theories. Theories were, after all, only robust and mature at the end of a long developmental process, and instructional design theories had a very short history. Lately, however, I have begun to believe that the discrepancy between instructional design theories and instructional design practice will never be resolved because instructional design practice will always be a form of situated activity (i.e., depend on the specific, concrete, and unique circumstances of the project I am working on).
Some other misconceptions I’ve seen:
- Online courses are worse than face to face learning. Some students are adamant about this belief. Obviously there are going to be bad online courses and bad face to face courses, it depends on contextual factors (the teaching, the students, the environment, etc.). But a recent meta-analysis from the U.S. Department of Education actually found that students learned more online than face to face, and the Open Learning Initiative is another example where students learn better and faster with their online statistics course.
- Simulations worse than the real thing. Again, of course there are some things you can only learn from doing the real thing (like some aspects of flying an airplane or fighting in combat). But study after study shows that you learn more and faster from a simulation than the real thing. I don’t have some of the references handy (but see this book chapter on how people learn with simulations I wrote), but it’s obvious why in most cases. A frog dissection simulation shows labels next to the body parts, for example. A flight simulator lets you rapidly re-practice tough techniques and challenging flying conditions. It’s not an either/or choice of course. You don’t want an airplane pilot who has only used a simulation, nor would you want an airplane pilot (unless very experienced) who’s never used a simulation.
- Lecture before simulation/experience. This is the belief that we should lecture students before letting them use a simulation or work on an open-ended problem, because they aren’t ready for the messy experience yet. This is counter-intuitive to most students as well, but the research shows that if you let students explore first, even if they make mistakes, and then lecture on the concepts (rather than the other way around), students will learn much more. Again, if you think about it, it’s not so counter-intuitive. If you get the lecture first, you more likely just tune it out like most lectures. If you get the messy experience first, you’ll start to formulate your own questions and ideas and so forth, and you are more ready and prepared to learn from the lecture that comes afterward.
- Topics should be broken into pieces linearly sequenced, with the learning objectives stated first, and going from simpler to more complex. This is related to the last one and there are numerous writings about this. Putting the learning objectives first isn’t bad, but it isn’t written in stone, either, nor are there learning objectives out there in the real world we are preparing students for. Almost 30 years ago Hermann Härtel wrote about why we still teach physics and so forth in a linear, piece-meal fashion, and problems with that approach. For example, often no connection is made between an electrostatics physics course and an electrical circuits engineering course, even though there are connections between the two. Chabay & Sherwood addressed this issue and Härtel’s philosophy with their Matter & Interactions curriculum and supporting software.
- If I tell it or have them read it, they should know it. This is the way most undergraduate courses still work. Weed out classes. If you didn’t memorize enough things, too bad, you fail. See below for 2 examples of where often times students didn’t even perceive or understand what you presented right in front of their eyes.
- Books and lectures are enough to learn anything – technology is not important, only teaching. As I wrote about in an earlier post on 50 Examples of the Need to Improve College Teaching, the National Center for Academic Transformation found that “Successful course redesign that improves student learning while reducing instructional costs is heavily dependent upon high-quality, interactive learning materials.” That means software. It can help teaching and learning. See the Open Learning Initiative I linked to earlier – one key to their success was the use of interactive statistics software. More and more modern topics are not so easily conveyed in textual or verbal form. And no that doesn’t mean ignoring teachers or teaching. Teaching IS a technology itself, as are books and lecturing and whiteboards.
Web design and instructional design (and other forms of design) appear to be unrelated, but the misconceptions of all them seem to largely stem from not understanding the context of design, which in many cases is the users or people, which means better understanding how people learn and how they think and perceive.
Web Design
I know of countless ‘mistakes’ or ‘errors’ students make when learning web design and HTML and CSS and so forth (forgetting that closing tag, etc.), but I’ll just mention one possible misconception (or popular myth) I see referred to on even professional web design blogs and sites, although I think there are hundreds of misconceptions, including a whole class of misconceptions about Web 2.0 versus Web 1.0 (web sites aren’t merely “electronic brochures” anymore, for example):
- “Click here” is bad – Yes, having a link that simply says click here is bad. The link needs to include context – what is at the linked site? But the blogs giving web design advice (such as this one) go too far in saying that “click here” should be completely avoided in favor of more abstract terms like “read more”. Using action verbs (like “click”) that tell people what to do IN ADDITION to providing some context leads to greater actual clickthrough rates. There is some research to support that, although I don’t have the reference at hand, but it shouldn’t be surprising – action verbs that tell people what to physically do result in (surprise) them more likely doing it.
Improving Design by Better Understanding How People Learn, Think and Perceive
The above was another example where some basic understanding of research on human learning, cognition, and perception can enhance your design (when the context of that design involves people as it commonly does in most areas). Of course, misconceptions about learning, cognition, and perception is a whole other topic that has not been well researched, but see this previous post on “Powerful Demonstrations about the Nature of Perceiving, Learning and Understanding” for some examples, such as change blindness (where something happens right in front of your eyes but you don’t see it). Just because you put that fact on your powerpoint or you stated something out loud to your students or you put some message on that web page or online course, doesn’t mean the students or your users or audience even perceived it, yet alone understood it or memorized it. We only pay attention to what we think is important, nobody notices the details, nobody reads the instructions.
Understanding misconceptions about design can reciprocally facilitate our understanding of teaching and learning, too, since instructional design is fundamental to education. Despite 30 years of overwhelming research on misconceptions in science education, for example, physics and sciences class teachers still often depend on classroom demonstrations to convey principles. See the article Why may students fail to learn from demonstrations? by Wolff-Michael Roth for an example of why that may not be such an effective instructional design technique. There is a gap (some have called the “valley of death“) between the research and the practice (of the designers, the innovators, the teachers,…), partly due perhaps to our misconceptions about design and implementation, such as “design arrogance” and “ideation without substance.”
Platforms for Developing Open Educational Software
Below is just a basic list of what software tools are popular now for creating various types of applications. I only post it because it changes from year to year and not everyone keeps up with this kind of stuff. I was also interested to see how well the Java platform could serve as an all-purpose open source stack for developing all kinds of educational software. It’s not always the best or most popular solution for some types of applications, but it does have the widest presence overall. Of course Sun, which controls Java, is being bought out by Oracle, which may have an impact on some of the below options (like perhaps project wonderland or javafx). I have nothing against .NET either (and esp. its open source clone, Mono). In fact for a good while I was the 2nd most active contributor to the Boo programming language and compiler that ran on the .NET/Mono CLR platform. But the fact that it still can’t do browser applets is a big strike against it for my purposes of creating educational software. Silverlight (and open source clone Moonlight) are making progress but still not there. Flash/Flex and Java are still the two best options there (with HTML5/Canvas being another up and comer). Google’s Java-like platform (such as Android) is becoming more dominant as well. They just released O3D, a new 3D multi-user browser plugin, replacing their earlier discontinued Lively tool. However it only runs on Windows and Mac and is still in early development. If they released a browser plugin for running Android applications they’d almost have the total stack as well, but I have a feeling Sun/Oracle wouldn’t be happy about that.
I didn’t have time to hyperlink everything – you can google to see the various tools in more detail.
- Learning Management Systems
- Moodle (PHP)
- Sakai (Java)
- Web-based application frameworks
- Rails (ruby)
- Grails (groovy)
- Turbogears, etc. (Python)
- Cake, Symfony, etc. (PHP)
- JSP, Struts, Velocity, Tomcat, etc. (java)
- ASP.Net (.NET/Mono)
- Cloud computing services
- Google Appengine (Python, Java)
- Amazon EC2
- Sun cloud
- Content management systems
- Drupal (PHP)
- Joomla (PHP)
- Plone (Python)
- DotNetNuke (C#, .NET/Mono)
- Browser-based Ajax/Javascript frameworks
- GWT
- JQuery
- Mootools
- Browser applets
- Flash/Flex
- JavaFX, Java
- Silverlight (.NET) Moonlight (Mono)
- 3D multi-user virtual worlds
- Second Life / OpenSim (.NET/Mono)
- Project Wonderland (Java)
- Croquet (squeak/smalltalk)
- O3D (Javascript, new from Google)
- 3D games
- Java – JMonkeyEngine, JOGL, LWJGL, etc.
- C/C++
- .NET – XNA
- 2D games/simulations
- Netlogo, Starlogo, Repast (Java)
- Pulpcore (Java)
- Processing (Java)
- PyGame (Python)
- Cell phone applications
- Android (uses java-like language and runtime by google)
- iPhone (apple)
- JavaFX mobile
- Java ME
- Desktop applications
- .Net/Mono System.Windows.Forms
- Java Swing
- C/C++ – GTK, QT
- Python – PyGTK, wxPython, etc.
Recent Sightings
These are from my bookmarks on del.icio.us:
- Software / Technology
- Quick-R is a help site for those using the R statistics tool instead of SPSS/SAS.
- Hippocampus is a homework and study help site with multimedia resources and course materials.
- Road Ready Teens is a program that includes a free driving videogame. See this article.
- First look at Ubuntu 8.04 Hardy Heron – the nest version of Ubuntu due out soon
- A report on open source video editing in Linux.
- Skype 2.0 for Linux has been released.
- New Classroom-Friendly Mini-Notebooks from HP
- Another great video on using and developing 3D educational worlds with Edusim in (Croquet-based) Cobalt. Here is more info on the Croquet-based Cobalt application used, and see also this tutorial for building a simple world in Croquet by Matt Schmidt.
- A video preview of Alice 3.0, the 3D programming environment
- Bionic Arduino – incredible course/workshop on creative uses for the Arduino microcontroller board.
- Adobe Photoshop Express Online – an online version of the image editor. I haven’t tested to see if it is any better than Picnik or other editors.
- More info on the next version of Java (update 10) and the new java plugin. See the article: Are Applets Back?
- OpenOffice.org and MS Excel: What’s the Difference? – There are some little differences that may get you, especially the use of semicolons instead of commas for passing parameters to functions. You get a cryptic “#NAME?” error if you use commas.
- Mahara 1.0 has been released – an open source electronic portfolio tool that integrates with Moodle
- On the quest for better screencasts – an interesting idea to make screencasts searchable and seekable via subtitles
- An overview of the samples that come with the Netbeans Java IDE.
- Amberjack is a free tool for creating guided tours of one or more websites.
- A new pdf book on the Scala programming language that runs on the java platform: Scala by Example
- Yahoo Announces Open Search Platform – Google discontinued their API for using their search engine from 3rd party apps long ago, so this is welcome.
- The Best Tools for Visualization
- DJ NativeSwing: JWebBrowser, JFlashPlayer, JVLCPlayer, JHTMLEditor
- Sites / Articles
- The call for papers for the 2008 Open Education Conference has been released. The submission deadline is May 30th.
- Multimedia Authoring Tools: The Quest for an Educational Package
- Video Research in Education describes a free pdf report from NSF on using video in educational research, which also was the basis for the book video research in the learning sciences.
- Grounded and Transferable Knowledge of Complex Systems Using Computer Simulations with links to the netlogo models used
- States’ Data Obscure How Few Finish High School
- Celebrating Seymour (Papert) – an article by David Shaffer on Seymour Papert’s legacy
- Numbers Guy: Stanislas Dehaene and Number Sense
- So Much for the Information Age – an interesting commentary at chronicle.com
- Educational Technology Magazine (pdf) special issue on open education resources
- The Truth According To Wikipedia – a video commentary on Wikipedia (via Larry Sanger)
- Becta Emerging Technologies for Learning Report
- A MacArthur series on Digital Young, Innovation, and the Unexpected
- An interesting article on Places to intervene in a system
- Multimodal Learning Through Media: What the Research Says – another positive lit review from the metiri group
- Cognitive Phenomenology: Marriage of Phenomenology and Cognitive Science
- Computational Thinking and Thinking About Computing – presentation with interesting examples of computing in everyday life by Jeannette Wing
- Computer Science Enrollment is Going Down
- From Java Platform Improvements to Better Teaching
- Robotic drumstick keeps novices on the beat
- The proceedings of the Tangible and Embedded Interaction Conference have been made available. See also the Symposium on Haptic Interfaces.
- Fritzing – a physical prototyping tool
