Thursday, May 14, 2009

Best Audio Format for iPhone Audio Programming

I had never done audio programming before I started my iPhone programming. After starting iPhone programming, I started to learn CoreAudio Framework, Audio Unit for Mac OS X system and iPhone System, and largest problem is which audio format should I choose for best practice?

After iPhone OS 3.0 beta was released, Apple finally introduced AVAudioRecorder class to AVFoundation framework. With which, we can play and record audio more efficiently! For me, I can cut the original playback code down from 400+ lines to 20 lines. And for recording, cut down from 300+ lines to 30 lines.

When testing AVAudioRecorder, the file format will significantly effect the file size. I had tried all the available data format when packed with .CAF file.  Here is the result:

[Updated@2009-05-25] YES, we can record with this format on simulator, but we CANNOT record this on the device! Remember this. While another one we can refer to is Apple’s official QA1615

[Updated@2009-06-15] Please do not use AAC if you want to play a system sound.

[Updated@2009-09-30] iPhone 3GS support AAC recording since iPhone OS 3.1 was released on Sept 9th. You can use hardware assisted encoding to record the AAC formatted audio file. But let me remind you, you can not use AVAudioRecord to record AAC audio file yet. The only option is using the RemoteIO Unit.

  • kAudioFormatLinearPCM               = 'lpcm',    OK    20.2M    .CAF
  • kAudioFormatAppleIMA4               = 'ima4',    OK    2.7M     .CAF [Best Choice]
  • kAudioFormatMPEG4AAC                = 'aac ',    OK    968K     .CAF [Updated 2009-05-25], [Updated 2009-09-30] Best Choice for iPhone 3GS
  • kAudioFormatMACE3                   = 'MAC3',    NG            .CAF
  • kAudioFormatMACE6                   = 'MAC6',    NG            .CAF
  • kAudioFormatULaw                    = 'ulaw',    OK    5.1M     .CAF
  • kAudioFormatALaw                    = 'alaw',    OK    5.1M     .CAF
  • kAudioFormatQDesign                 = 'QDMC',    NG            .CAF
  • kAudioFormatQDesign2                = 'QDM2',    NG            .CAF
  • kAudioFormatQUALCOMM                = 'Qclp',    NG            .CAF
  • kAudioFormatMPEGLayer1              = '.mp1',    NG            .CAF
  • kAudioFormatMPEGLayer2              = '.mp2',    NG            .CAF
  • kAudioFormatMPEGLayer3              = '.mp3',    NG            .CAF
  • kAudioFormatAppleLossless           = 'alac'        OK    4M        .CAF
  • kAudioFormatMPEG4AAC_LD             = 'aacl',    NG            .CAF
  • kAudioFormatAMR                     = 'samr',    NG            .CAF
  • kAudioFormatiLBC                    = 'ilbc',    NG    Wierd    .CAF

The list here has omitted all the other data format that are not supported by CAF file extension. You can use the console application afconvert to verify each supported file format and data format via typing this:

afconvert -h 

All the tests were recorded in 60 seconds, with sample rate@44100, 2 channels, 96kbps. The RecordSetting Dictionary object is used as below:

recordSetting = [[NSMutableDictionary alloc] init];

//General Audio Format Settings (Necessary for all audio format.)
[recordSetting setValue:[NSNumber numberWithInt: kAudioFormatMPEG4AACkAudioFormatAppleIMA4] forKey:AVFormatIDKey];
[recordSetting setValue:[NSNumber numberWithFloat:44100.0] forKey:AVSampleRateKey];
[recordSetting setValue:[NSNumber numberWithInt: 2] forKey:AVNumberOfChannelsKey];

//Linear PCM Format Settings (only necessary when you want to record Liner PCM format)
[recordSetting setValue:[NSNumber numberWithInt: 32] forKey:AVLinearPCMBitDepthKey];
[recordSetting setValue:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsBigEndianKey];
[recordSetting setValue:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsFloatKey];

//Encoder Settings (Only necessary if you want to change it.)
[recordSetting setValue:[NSNumber numberWithInt:AVAudioQualityMin] forKey:AVEncoderAudioQualityKey];
[recordSetting setValue:[NSNumber numberWithInt:96] forKey:AVEncoderBitRateKey];
[recordSetting setValue:[NSNumber numberWithInt:16] forKey:AVEncoderBitDepthHintKey];

//Sample Rate Conversion Settings (Only necessary when you want to change the sample rate to a value different to the hardware sample rate, AVAudioQualityHigh means no conversion, usually, 44.1KHz)
[recordSetting setValue:[NSNumber numberWithInt:AVAudioQualityHigh] forKey:AVSampleRateConverterAudioQualityKey];

[Updated 2009-09-30], The record settings listed above was for example only, if you want to record IMA4 or other compressed audio format, you are no need to put the Liner PCM part. Other parts are commented for their purpose.

You can see, the MPEG4AAC has the minimum file size and same quality when compare to other file formats. About the AAC format, please refer to [Wikipedia – AAC"]

Apple, you could just give us a best practice guide! So I won't need to test it one by one myself. OK, I know, you are working out of people, hire me, I would love to do these works.

24 comments:

  1. Using AVAudioPlayer, I can't get MP4 AAC audio to play in my iPhone app. Are there special encoding parameters that need to be set to make this happen? (I've even tried saving the audio as a .caf file with AAC encoding; that doesn't work.)

    Thanks.

    ReplyDelete
  2. Greg,

    I don't know are there any special encoding parameters needed.

    I just simply use the afcovert command in the console and even did not adjusted the sample rate.

    My command:
    afconvert -f caff -d 'aac ' 01.caf 01-aac.caf

    That's it, and it worked fine with AVAudioPlayer.

    Did you tried to use another test audio file?

    ReplyDelete
  3. When I use 'aac ', I lose the ability to play two sounds at the same time. Is that what you observe?
    /usr/bin/afconvert -f caff -d 'aac '

    Also, when I use 'aacl' the sounds do not play and no error message seems to be generated.
    #/usr/bin/afconvert -f caff -d 'aacl'
    Again, do you see the same issue?

    Thanks.

    ReplyDelete
  4. Never mind. I see in Apple's docs that AAC is implemented by hardware codec.

    ReplyDelete
  5. Yeah, you are right, maybe you should multichannel mixer to play 2 AAC file simultaneously.

    ReplyDelete
  6. Hi Tony,
    do you know if we could record audio directly to mp3 format? Thanks!

    Ken

    ReplyDelete
  7. No, you can't, because of the lack of CPU process speed. And thus apple did not provide any support for mp3.

    ReplyDelete
  8. Hi,


    I am develooping an application which records voice and sends the file to the server using webservice.
    If i am using 'kAudioFormatAppleIMA4 ' what should be the filename used to store the file(extension).
    Can we not use .CAF?
    I tried appleLssless using .m4a extension...

    Please help..

    ReplyDelete
  9. I don't know if I understand you correctly. If you are using AVAudioPlayer, I suggest you do not change the file extension, or only change it on the server side. When you download it, change it back to CAF.

    I didn't have a try, just guess. Basically, you cannot change the file extension, because a lot of functions are based on the extension name, change it means you ask the system to treat it as a different kind of file.

    ReplyDelete
  10. HI, thanks for the reply...

    anyway now i am recording the audio using kAudioFormatAppleIMA4 and saving it in a file test.aiff.
    Then i upload it to server.

    The only issue is the file size is too high....

    In blackberry they support recording in amr format which is very light weight. But aiff format is almost ten times heavier... other formats are even more bulky...

    Any other alternatives???

    ReplyDelete
  11. Tonny,

    I read your blog on audio formats and the documentation you provided. You also mention on the last line about hiring you. What is your availability for the next couple of days. We have an app that is beta testing. We used a modified speakhere set of classes. It did the job in a bad way, the file sizes are enormous therefore it is flagged as a bug from our client. I have just started using the AVAudioRecorder. It is a lot easier to work with, and alot fewer lines of code. It is recording in linear PCM as .caf file. I personally don't mind that. Our client has requested .m4a or .mp4 format. Does this sound like something you would be interested in? Contact me via email.

    thanks.

    ReplyDelete
  12. Hi Tony,

    Your sample code above for settings seems to have the format setting as MEP4AAC but its not supported on the iPhone.

    Also, it contains settings for Linear PCM. Are these settings for linear PCM required if the format is not linear pcm.

    Thanks,
    Aashish

    ReplyDelete
  13. Hi, Aashish,

    Yes, you were right, I placed AAC setting in the code, that's wrong, I will correct it. Thank you.

    No, Not all the settings are necessary for LinerPCM. I will added some additional information to the LinerPCM settings tomorrow. Hope you can come back and see it. Thank you again.

    ReplyDelete
  14. Tonny,

    Will wait for the modified settings.

    Tx,
    Aashish

    ReplyDelete
  15. Hi Tonny,

    The new settings with IMA4 work great. One issue seems to be that the file created is only playable on the iphone. I've tried different extensions aiff, aif, ima4 but can't get the file to play on any desktop Quicktime player.

    Thanks,
    Aashish

    ReplyDelete
  16. Hi,
    i know it is a bit weird question/task..
    i would like to record in 50 kh, however play the voice i just
    recorded in 20 kh is it possible? how can i tranfer the rates?
    and is it possible to record so hi rates (50 kh)?
    thanks
    dan

    ReplyDelete
  17. Hi,,Is there any way of record audio in .mp3 format..Or how we can convert ptogrammicaly any .caf file to .mp3....And the MAX size of uploading audio file to serve......

    ReplyDelete
  18. Hi,,Is there any way of record audio in .mp3 format..Or how we can convert ptogrammicaly any .caf file to .mp3….And the MAX size of uploading audio file to serve……

    ReplyDelete
  19. I am not new to blogging and truly appreciate your site. There is much prime content that peaks my interest. I am going to bookmark your web site and keep checking you out.

    ReplyDelete
  20. No, there is no direct way to create mp3 file. You can compress the audio data into mp3 using LAMP or some other encoders.

    ReplyDelete
  21. Thank you, Curtis! I will keep working on this. Thank you for your support!

    ReplyDelete
  22. Hello
    I try to use AVAudioRecorder with your exemple but I don't hear sound when I played with AVAudioPlayer (test on simulator).
    It's not worked on simulator?

    ReplyDelete
  23. [...] iPhone – Best Audio Format With AVAudioRecorder, we can create different audio files in different format. So how could we determine which format we should use? i found a blog post which could give you the answer. The author Tonny has tried different kinds of audio formats and you could find the results at Best Audio Format for iPhone Audio Programming. [...]

    ReplyDelete
  24. Best write-up, I’m a enormous believer in commenting on blogs and forums to assist the website authors realize that they have added anything of value to the world-wide-web!

    ReplyDelete