Mail.app and iPhone encode Japanese as Korean

The latest of ridiculous bugs I have found after my switchover to Mac is that Korean takes precedence over Japanese when “Automatic” is selected as the encoding on Mail.app. It literally sees Japanese text as Korean and encodes emails as ISO-2022-KR (Korean) if both Japanese and Korean are selected under Languages.

200807271725.jpg

Try it yourself: make sure both Korean and Japanese are in the Languages list (so they show both show up in Mail.app’s Message > Text Encoding) and send an email containing Japanese text when encoding is set to “Automatic”. View the full message source and see that it encodes the message as ISO-2022-KR, not ShiftJIS or UTF-8 which it should. You have to completely remove Korean from the list if you want to use Automatic encoding, or manually select the encoding yourself every time. I cannot find a setting to force an encoding (er, UTF-8) on every email you send.

This is not a problem if the recipient is a PC/Mac user since the mail client will sort it out – but sending mails to a Japanese phone, which obviously has no idea what to do with Korean text encoding (and can’t suss out that its actually Japanese text) means that the lucky recipient gets 文字化け、mojibake, jumbled garbage.

Its even worse on the iPhone

If you ever decide to turn Korean keyboards on in the Language settings, or switch to Korean, iPhone appears to add Korean to the Languages list when choosing its own encoding. However, you cannot change the encoding manually on iPhone, or ever remove Korean from the list, even if you then disable Korean input.
The net result? You can never send Japanese mails to Japanese mobile phones ever again on your iPhone, until you do a full system restore. Which I am now doing. If you reply to a mail that was sent to you in ShiftJIS, the iPhone is at least clever enough to reply in the same encoding type but you are SOL if you want to compose a new one.
Rubbish. Windows Mobile has an option to “always send mail as UTF-8”. I wonder how many iPhone users in Japan have fallen foul of this?

iPhone improvements for the UK and Japan

El Reg is reporting that O2 has been told by Apple not to release sales figures until Apple say so.

UK carrier O2 has confirmed that Apple has placed it under a gagging order to prevent it from publicly revealing how many iPhone handsets it’s sold to date.

The company’s head of media relations, Simon Lloyd, told Register Hardware that it’s a stipulation within the company’s sales agreement with Apple that O2 can’t release any such details until the Mac maker says so.

Lloyd would only say that O2’s UK iPhone sales in the two weeks up to Christmas period were “in line” with its expectations.

This is because the iPhone is completely bombing in the UK. Being in a University environment (where students always like tech gadgets), the only iPhone I have seen in the wild is an unlocked hacked-to-bits US model. The iPod Touch is undoubtedly a sound investment at just shy of £200 if you want a taste of “next gen” touchscreen interfaces and you are far better off buying an iPod Touch for the fancy new iPod bits and getting a free phone on a cheaper contract. You would have to be a complete mug (or rich fashionista) to buy an iPhone at £269 and be locked into an 18-month contract with O2 at £35 a month.

If the iPhone actually did half the things standard UK mobiles have done for years, it would maybe be an acceptable high-end phone model, but its not. The US mobile industry is a couple of years behind ours (people import our Nokia N95s for instance) so the iPhone looks like a fantastic bit of kit in the US.

The funniest thing is that the iPhone will be on DoCoMo in Japan soon – and it will need a complete feature overhaul to be even classed as a ケイタイ (keitai, mobile phone) out there, let alone here. To work in the Japanese market, Apple need to add:

  • 3G support. GSM/EDGE/GPRS do not exist there.
  • MMS support for their mobile email with emoticons (絵文字, emoji). Without emoticons, users will get gibberish when recieving mails from “proper” mobile phones in Japan.
  • Java application downloads for iAppli
  • Flash Lite for the on-demand games that are springing up on the Japanese mobile internet
  • QR barcodes. Apple should be pioneering these in the rest of the world already with their clout, but aren’t. The possibility is enormous here – in a music magazine, a QR code could be next to a new album review: the user scans it and can buy it from iTunes immediately over the air, no text input required. This happens in Japan already for all sorts of mobile content – Apple could be making this popular everywhere.

The Japanese text input is already programmed and is damn good on the iPod Touch, so that doesn’t need doing. Stuff like IC chips aren’t required, so its only those five points that I cannot see Apple surviving without. Windows Mobile phones released in Japan on Softbank now get an application for MMS with emoticons because the original models were seen as pretty basic without it.

Sort it out Apple!

PocketStackz (PocketPC) review

For Japanese language practice for my course I use PocketStackz by some chaps called “Minddate software”. Its an all-purpose language flashcard program with an emphasis on Asian languages (such as that an Asian Unicode font needs to be installed on your PocketPC).

What sets it apart is that you can immediately see what words you need to study at any point since the program sorts the words into “stacks” from “unknown” on the left to “known” on the right. If you get an answer wrong when testing yourself, the word moves towards the left, and visa versa if you get one right. This means when you have only a few hours until a vocab exam, you can quickly refresh the “most unknown” vocab in the quiz.

Vocab or kanji can be tested in pretty much any combination (romaji to kana, kana to kanji etc) and I find I get better results if I use it for short periods of time often. My test results even confirm this – weeks where I’ve spent 5 minutes a day with PocketStackz have seen perfect vocabulary test scores. You still need a drive to study, when when you don’t need to get your textbooks and pen and paper out (or those silly little “word cards”) its easier to find the time.

The software comes with a free PC application so you can make your own lists of vocabulary, but files for many textbooks and series of kanji (including the official joyo kanji) are available on the developers website. I had to make the kanji files for the Genki textbook series since they were not available, but this was painless using the PC software. Top stuff. 5/5.

Plus its only 19$ from Handango and they have a free trial (which I liked so much I bought it). Thats £8!

Buy or trial PocketStackz from Handango

Genki Japanese Textbook Vocab file for PocketStacks
Genki Japanese Textbook Kanji file for PocketStackz

Japanese keitai flashcards

Update April 2012: Five years later after several server moves, this is no longer online – I’m still hunting for the files. Most likely it wouldn’t work on new keitai’s anyway.

Edo no Tango (Ed’s Tango, エドの単語) is a Flash Lite based Japanese language vocabulary and kanji flashcard utility. Developed to help my study, it currently contains lessons 10 through 17 from the Genki series of Japanese language textbooks.

It uses Flash Lite 1.1 so runs on the vast majority of Japanese contract phones (the pre-paid ones can’t quite manage it). Give it a try! Press the 1 key when in the flashcards to display the button commands.

Postmortem

Flash Lite 1.1 is extremely basic – it only has the features of Flash 4. Flash Lite 2 is much better, supporting a Flash 7-esque feature set but at the time of creating the program was not on any Japanese phones yet – the closest you could get was a downloadable plugin for Nokia devices. There are several limitations of Flash Lite 1.1 which made development challenging:

  • No Unicode support. This was the first major hurdle. Even on Japanese devices, it is impossible to display Japanese text using the device font. Flash Lite 2 fixes this, but in the mean time Japanese text needs to be encoded as vectors or images.
  • 100kb network limit. It makes sense, but all the Japanese networks limit the size of any one file you download to around 99kb. This means with no Unicode, images or vectors for each kanji character soon add up.
  • Dynamic network retrieval of data is limited to rudimentary text files with no XML support. No Unicode means retreiving data was impossible anyway, but this meant I had to create a seperate file for each set of flashcards, needlessly increasing data use and not making a very elegant solution.
  • The phones have very limited memory, so my original idea of using 2bit images for each flashcard resulted in graphical corruption every time. Images would just turn red after showing four or five of them. In the end, 2bit images were converted into vectors using the “break apart” command.

I don’t need to use this now (I bought an off the shelf Pocket PC application to do the job) so I haven’t got round to adding new lessons. Its not simply a case of putting new data on a server unlike a Flash Lite 2 based version would be capable of.