Open Sourcing “Cognition”

One of the perks of working in R&D at Medidata is “Innovation Time”. Twice a year, engineers get to spend two weeks building whatever they want. Ideally some of the things we build will go back into direct product development but if not, we are encouraged to open source the work.

Fresh from attending Microsoft’s Build 2013 conference in San Francisco, I wanted an excuse to use all the latest Microsoft web stack tools. Namely:

  • Visual Studio 2013 Preview
  • ASP.NET MVC5 Beta
  • Entity Framework 6 Beta
  • SignalR 2.0 Beta

I also wanted to test out CouchDB, which was made infinitely easier with the awesome MyCouch library by Daniel Wertheim. The result of which is “Cognition” – a cross between a wiki and a CMS that can be used as a basic CRM, a job board or knowledge base. The result of two weeks work can be seen on the company repo at https://github.com/mdsol/cognition – a test site can be seen hosted on Azure at http://cognition-demo.cloudapp.net. There is quite a lot of rough code with scant test coverage, unfortunately necessary to get the experiment out the door in two weeks. It does however show what is possible with the .NET web stack.

imageimage

Here is a short video showing the current state of the app (no audio – visit the full post page to view): Read more

Working on an online MS Money replacement

I am finally fed up of having to install Microsoft Money and set up syncing of the data file, so I’ve decided to try and build a web based version. Mint.com does not support non-US markets and apparently does not even support adding your own transactions that have not been downloaded from a bank. “Private Money” is the codename and it looks a bit like this at the moment:

Capture

The stack:

  • ASP.NET MVC 3 as the application framework
  • Entity Framework 4.1 for ORM
  • NInject for a modular design with dependency injection
  • MigratorDotNet for database versioning with some tweaks to get it to work over different modules
  • jQuery and jQuery UI for some nice buttons, animations and AJAX loading. MVC 3 has great support for unobtrusive javascript with jQuery which is a dream to use.

I am planning on replicating the parts of MS Money that I use the most, mainly the transaction logging, reports and cashflow chart (which should be fun to make on a canvas element). Visually I am going for the Windows Live/Metro motif.

Avatar Kart – now with avatars!

My Avatar Kart game now has:

  • A skybox
  • An original track
  • a floor for the world
  • Debug and chase cameras
  • A starting countdown

I also finally got Avatars in the code (Xbox 360 only so no videos yet). Microsoft has not decided to release the bones for the avatars, so I cannot do custom animations or poses yet. Therefore they are standing in the karts at the moment…

image image

Avatar Kart work in progress

During my down time I’ve been cracking on with XNA and am developing “Avatar Kart” – essentially Super Mario Kart on the SNES with Xbox avatars (we have to wait until XNA Game Studio 3.1 this summer for avatar support). In a couple of days I’ve managed to finish:

  • CPU Car AI
  • Physics and wall collisions
  • Terrain effects
  • Lap counts and car rankings
  • Split screen

The track is still from SMK and the cars are empty but it does the job. Although its in 3D, I’m trying to get a 2D mode7 type handling feeling, really tight controls and simple graphics. I’ve already hit the performance wall of floating point operations in the compact framework on the Xbox so will need to spend some time optimising later. I will be replacing the Mario Kart track with my own design soon.

Below is a video of how its looking so far:

Fullscreen flashcard program

image I’ve put together the first version of a simple program for displaying flashcards in fullscreen, mainly as an exercise in WPF but also because its damn useful for all sorts of teaching environments. Flash cards are simple text files that anyone can edit – just load them up and hit spacebar to cycle through them. “Big Flash Cards” is now at version 0.1 🙂

It supports Unicode so is perfect for Japanese lessons, for which I designed it. Example flash card files are included – they are very easy to edit and are simple text files. This is a very early version so there will be bugs.

bigflash

Download Big Flash Cards (117kb zip)

Vista and Windows 7 users: just unzip and run BigFlashCards.exe
Windows XP users: You need at least version 3.0 of the .NET Framework. Download it here.

For those that care, this was developed in Visual Studio C# 2008 solely on a tiny little Dell Mini9 laptop.

Lines EX – rebuilding with XNA 3

My dissertation project saw me making an XNA game (Lines) and website to go with it (edngames.com) which used web services to make essentially a cut down Xbox Live on the PC with rich presence support, automatic high score uploading etc. I feel it was very successful (and the markers agreed) but I didn’t get the time to spend on the game portion that I wanted to, instead concentrating mainly on the distribution scenarios and the user experience. With the Xbox Live Community Arcade coming up and the new CTP of XNA 3 released, I thought it would be the right time to rewrite the game using everything I’ve learnt over the last year since I began the project ready for the Community Arcade launch “this fall”.

image

Instead of using SpriteBatch, I’ve decided to place 2D flat quads in 3D space. While still getting free rotation, transparency etc, I now get resolution independence and extra effects such as being able to skew sprites.

I already have the main game code so am working on the menu system. By having sprites in a 3D space, changing the camera location means aspect ratio independence too by zooming out or changing the field of view – below shows the main menu in 16:9 and 4:3 (the zooming will work with all aspect ratios meaning no stretching).

image image

My favourite benefit however is the Smash Bros. style 3D effect you can achieve by moving the camera with the right stick. Completely pointless but very cool and can help you debug the locations of your sprites in the game world.

image

And yes, I am aware of the hidden word in the title and it is purely accidental 🙂

Symfony in Enterprise – Tips and Experiences

200807021841.jpg

At work I am currently tasked with redeveloping an intranet application used to track customers, products owned, support contracts, support records and all sorts of other CRM-esqué functions. The version used at the moment is a very fast Perl/MySQL/Mod_perl/Apache setup built over a few years. Its got to the point where the company needs more functionality and hacking extra functions into the current code is getting more and more difficult.

Why Symfony? Why PHP?

The current trend for “rich internet applications” is Ruby on Rails. Ruby has very powerful Object Role Modelling features that completely abstract the database from your code – no more writing SQL queries once the initial database has been set up. Database tables become class generators and rows become class references, free to be instantiated, updated and saved. Symfony is probably the closest PHP Framework to Ruby, as it utilises the Propel ORM layer (so can use almost any RDMS) and has a full Model View Controller structure. PHP runs on the vast majority of Apache installations and requires no extra software installed on the server – in addition finding skilled PHP programmers is far easier (although this might change). Symfony is also probably one of the most well documented Open Source web framework projects that I have come across, featuring a whole book written by a member of the community very close to the development team, an expansive Wiki on the project homepage and hundreds of plugins to simplify everything from AJAX to RSS feed generation.

Symfony can be used in two ways – through config files (text files defined using YAML, a simple markup language) which are passed to Generators to create the final scripts, or by manually coding the actions and pages. In reality, a combination of the two will be used. Tweaking the included Administration Generator will yield respectable results for the standard CRUD (Create, Read, Update, Delete) functions and it is extremely simple to create modules and actions with the included command-line tool that creates skeleton templates for you to use.

No longer do you have to plug in debugging tools (such as the Xdebug php extension) to get decent errors and stack traces when your application fails – Symfony provides a development view of your application that will output extremely useful information (such as execution time, a list of all SQL queries made and globals) right in the page outputted to your browser. Debugging the application I’ve been working has been as easy as that of a desktop app (coming from my experience of Visual Studio) – Symfony will work in conjunction with many PHP debugging extensions for further information if you need it (I haven’t).

Plus, Symfony is good enough for Yahoo to use it for a 20 Million user app, Yahoo Bookmarks!

Results

I’ve been working alone for a month on the project and what strikes me is the huge amount of already achieved with no prior experience of Symfony:

  • A complete database redesign using the excellent MySQL Workbench. Adding many-to-many relationships where there were none before requires a good deal of thought but the results when the ORM classes are generated by Propel in Symfony are well worth the effort
  • Database migration handled by the CLI features of Symfony. One command will generate a .php file you can run at the command line (or cron job) with automatic access to all the features of your main application. Where the database had been vastly redesigned to support relationships, custom migration code had to be written (but where tables have not changed, you can copy table data directly in one line of manual SQL). The migration script takes about half an hour to run (approx. 3,000,000 records).
  • User security and session support, timeouts etc
  • Full creation and editing of all the major record types with administration control panels
  • AJAX views and manipulation of information with per-user rearrangeable panels ala iGoogle
  • Full filtering functions for searches with auto-paginated results (soon with Excel export)
  • Global per-user filters that effect all searches
  • …and lots of other actions specific to the application

Tips

Some advice to newcomers to Symfony from my experience:

  • Don’t rewrite the wheel. Use the plugins available and tweak them if necessary.
  • Use the Admin Generator, don’t fight it. Instead of writing custom create/edit/list actions when the Administration Generator does not meet your requirements (or altering the CSS won’t help), override templates with per-module versions of your own. Even better is creating a new Generator all together (although it is a headache writing PHP code that writes PHP code!) – check out the sfAdvancedAdminGenerator plugin for inspiration.
  • Don’t worry too much about performance especially if your application is running a bit sluggish on your local machine. On a proper server with a production installation of Apache and MySQL, the application I thought was a bit slow flew given the chance. You can optimise later.
  • Don’t be scared of upgrading – I went up a symfony version during production without a hitch and without a change in my code (from 1.0 to 1.1 this will be different)
  • Use a decent IDE. I use PDT Eclipse and it definitely speeds up development. You will frequently have to edit several files at once so at least use a text editor with tabs.
  • Read the book! I can’t stress this one enough. It is extremely well written and is available for free on the website.

I’ll post some more impressions and a brief postmortem when the project is finished.

Tomoyo 0.5 theme for WordPress

UPDATE! This version is out of date, please see the latest verson on the main theme page here: //www.edandersen.com/projects/tomoyo-wordpress-theme/

 

EdAndersen.com - Windows Internet ExplorerI toiled away for quite a while to create this WordPress theme which I think is pretty unique. I’m offering it up to the WordPress-using world. Just unzip into your themes directory.

Download Tomoyo theme for WordPress (303kb zip)

Features include:

  • Sexy Flash-based headings using sIFR, for post and page titles and sidebar headings (degrades gracefully when no Flash or Javascript available)
  • Full WordPress Widget support with three sidebars to customise (the one on the right, and two in the footer)
  • No horizontal scrolling on a 800-wide screen (perfect for eeepc users!)
  • Uses the new Meiryo Vista font if available for seamless latin and japanese mixed text
  • Tested with WordPress 2.3

I still need to style the comment system and there are a few rough edges, but I thought I’d release it anyway. All you need to do it change the header image from my ugly mug and its pretty much your theme!

Japanese keitai flashcards

Update April 2012: Five years later after several server moves, this is no longer online – I’m still hunting for the files. Most likely it wouldn’t work on new keitai’s anyway.

Edo no Tango (Ed’s Tango, エドの単語) is a Flash Lite based Japanese language vocabulary and kanji flashcard utility. Developed to help my study, it currently contains lessons 10 through 17 from the Genki series of Japanese language textbooks.

It uses Flash Lite 1.1 so runs on the vast majority of Japanese contract phones (the pre-paid ones can’t quite manage it). Give it a try! Press the 1 key when in the flashcards to display the button commands.

Postmortem

Flash Lite 1.1 is extremely basic – it only has the features of Flash 4. Flash Lite 2 is much better, supporting a Flash 7-esque feature set but at the time of creating the program was not on any Japanese phones yet – the closest you could get was a downloadable plugin for Nokia devices. There are several limitations of Flash Lite 1.1 which made development challenging:

  • No Unicode support. This was the first major hurdle. Even on Japanese devices, it is impossible to display Japanese text using the device font. Flash Lite 2 fixes this, but in the mean time Japanese text needs to be encoded as vectors or images.
  • 100kb network limit. It makes sense, but all the Japanese networks limit the size of any one file you download to around 99kb. This means with no Unicode, images or vectors for each kanji character soon add up.
  • Dynamic network retrieval of data is limited to rudimentary text files with no XML support. No Unicode means retreiving data was impossible anyway, but this meant I had to create a seperate file for each set of flashcards, needlessly increasing data use and not making a very elegant solution.
  • The phones have very limited memory, so my original idea of using 2bit images for each flashcard resulted in graphical corruption every time. Images would just turn red after showing four or five of them. In the end, 2bit images were converted into vectors using the “break apart” command.

I don’t need to use this now (I bought an off the shelf Pocket PC application to do the job) so I haven’t got round to adding new lessons. Its not simply a case of putting new data on a server unlike a Flash Lite 2 based version would be capable of.