Always Use an Active Scheduler
This can't be stated enough the multimedia subsystem cannot work without proper use of an active scheduler and associated active objects. Some APIs for example, within ICL explicitly use TRequestStatus to show asynchronous behavior and expect the client to drive them via an active object. With others, the behavior is callback-based and its asynchronous nature is hidden. Virtually all the APIs make extensive use of CActive-based active objects behind the scenes and need an active scheduler to...
File Conversion
CMdaAudioConvertUtility shares many of the calls of CMda-AudioRecorderUtility. They both derive from CMdaAudioClip-Utility, although the reasoning for the common base class is now not very clear. The two classes cover very different use cases - CMdaAudioConvertUtility is typically used to copy from one file to another, with some format conversion action. It is different from the other APIs in this section in that it does not involve using the microphone or the speaker. Because of their shared...
Playing Tone Sequences
Playing a tone sequence is akin to playing an audio file see Section 5.3 but sequence files are inherently simpler. Not only is no controller required but the sequence is decoded in the adaptation layer. Although the use case is similar to that of playing audio files, the calling pattern is similar to that of playing DTMF or tones. There are several ways of playing a tone sequence provide the tone sequence in a descriptor provide a filename that contains the tone sequence data use RFile Open to...
Future Multimedia Support
In this section, we take a look at what the future holds for multimedia support on Symbian OS. Some features are not yet visible to a third-party application developer, as noted in the appropriate sections, and some may never be made accessible. However, being aware of their existence contributes to a better understanding of the multimedia subsystem. 2.5.1 Metadata Utility Framework MUF Symbian OS v9.5 adds a Metadata Utility Framework MUF to the multimedia subsystem. The MUF is independent of...
Evolution of the Multimedia Subsystem in Symbian OS
Having explained the background against which Symbian OS has developed, we're going to take a closer look at its multimedia subsystem and how it has evolved, in order to better understand its current design and use. Readers who are either familiar with the history or not interested in the technical details can safely skip ahead to Section 1.6 for a peek into the future If you're not already aware of the history of Symbian OS releases, then we'd recommend taking a look at a visual timeline which...
Controlling Screen Output Setting the Display Window
We saw in Section 4.5.1 that the CVideoPlayerUtility NewL method takes a number of parameters related to the display. The CVideo-PlayerUtility class also has a method, SetDisplayWindowL , which you can use to change those parameters later if you want to. void SetDisplayWindowL RWsSession amp aWs, CWsScreenDevice amp aScreenDevice, RWindowBase amp aWindow, const TRect amp aWindowRect, const TRect amp aClipRect Both the NewL method and the SetDisplayWindowL method take a window server session, a...
Opening the Video for Recording
You can open a file to record a video into using the OpenFileL methods void OpenFileL const TDesC amp aFileName, TInt aCameraHandle, TUid aControllerUid, TUid aVideoFormat, const TDesC8 amp aVideoType, TFourCC aAudioType void OpenFileL const RFile amp aFile, TInt aCameraHandle, TUid aControllerUid, TUid aVideoFormat, const TDesC8 amp aVideoType, TFourCC aAudioType You have the option of specifying a file name into which the video will be recorded or a handle to a file that is already open for...
Getting Information about the Opened Image
Our first attempt to open and decode the image used quite a naive approach to the destination bitmap size - we just used a square bitmap of size 20 by 20 pixels. The following methods of the CImageDecoder class can be used to obtain the most vital information about the opened image const TFrameInfo amp FrameInfo TInt aFrameNumber 0 const FrameCount is used to obtain the number of frames from which this image is made except for the special case of the MNG format, which is discussed in Section...
Trick Play Modes
Trick play refers to operations such as fast forward, reverse playback, and slow motion playback. Prior to Symbian OS v9.4, there were two ways that trick play could be implemented The video controller could supply a custom command2 that allowed these modes to be used see Section 4.15 for more information on custom commands . The SetPositionL method see Section 4.5.8 made it possible to jump forward or backward through the video by changing the position that the video is playing from, which...
Stream Calls for Recording
Like most of the audio APIs, CMdaAudioInputStream operates in two phases initialization and the action itself. The key operations are shown below RecordL is called by the application and the Maisc-OpenComplete callback is called from CMdaAudioInputStream itself. void CIOStreamAudio RecordL TInt aBufferSize if buffer is already allocated, throw away and re-create iMainBuffer.Close iInputStream- gt Open NULL use default settings The RecordL operation initializes iMainBuffer where we store the...
Audio Policies
It is sometimes tempting to forget this fact when writing an application, but Symbian OS is a multi-tasking system and there are always some other programs running. At various times, your programs may be running in the foreground or in the background - let alone the fact that some services are hidden but running continuously. Having said that, resources on a smartphone are always limited - particularly if special hardware is used to support audio codecs. Additionally, however, there are...
Camera Control
A client may not always be able to gain control of the camera. If more than one client is requesting control, then a priority calculation is used to determine which client should have it. Higher priority clients take ownership from lower priority clients when they contend for use of the camera. There are two parts that make up a client's priority value Clients may declare themselves to have a priority from -100 to 100. The default priority is zero. Clients with MultimediaDD capability take...
Error Handling
The permitted camera calls depend on the state of the camera. Illegal calls are made if We programmed incorrectly. We may repeat a call unnecessarily, such as by calling Reserve when the camera is already successfully reserved. The camera has been seized by another client and we have not received a notification, if we are using MCameraObserver rather than MCameraObserver2. If we make an illegal call using a method that can leave, the function leaves with the error code for example, calling...
Scaling Bitmaps
The following code demonstrates the basic use of the scaling API. It makes use of the single-bitmap variant of the API, specifies the destination size and asks the scaler to preserve the aspect ratio of the image CActiveScheduler Add this Add to scheduler CSimpleBitmapScalerHandler CSimpleBitmapScalerHandler Cancel any request, if outstanding Cancel Delete instance variables, if any const TSize amp aDestSize, CBitmapScaler TQualityAlgorithm aQuality We are not checking the error code of this...
Preparing for Video Capture
Before we can capture video, we need to prepare the CCamera object using the CCamera PrepareVideoCaptureL method. This allows the camera subsystem to allocate any memory necessary and perform any other setup required to capture video. aSizeIndex, aRateIndex, If video is not supported, then the function leaves with the error Video settings should be used for capturing video only for image capture, the camera should be prepared separately see Section 3.5 . Video capture cannot take place while a...
Direct Screen Access
One method that a controller or codec can use to send output to the display is called direct screen access. This allows drawing to the screen without having to use the window server. This is the default method of screen access prior to Symbian OS v9.5. A controller that wants to use direct screen access will start and stop it automatically as required. It is, however, possible for you to start and stop the direct screen access explicitly if you wish to do so. void StopDirectScreenAccessL void...
Platform Security and Capabilities
Symbian OS v9.1 introduced platform security features to stop untrusted or unsigned applications performing key operations. So the question often asked is 'will the platform security features stop me writing an application that plays or records audio ' The simple answer is 'not really'. Your application doesn't need any capabilities to play audio. There is a capability termed MultimediaDD that is required by some middleware, the adaptation and some key applications, but it is very unlikely that...
Basic Image Settings
The CCamera class allows access to the camera's settings. It provides functions that allow you to select the image format and to alter brightness, contrast, zoom, exposure, flash and white balance levels of the camera image. Before using such functions though, you should test that the camera supports what you want. To do this, get a TCameraInfo object using the CameraInfo function. The object's iOptionsSupported member is a bitfield of flags that describe which options are supported by the...
Interpreting Error Codes
Unfortunately real-world applications have to deal with errors. Errors may arise due to input data being invalid, inconsistent application requests to the API, implementation restrictions, and so on. The image decoder API uses normal system-wide error codes to let the user know if something goes wrong. There are no CImageDecoder-specific error codes defined. The most common source of error codes is the set of CImageDecoder factory functions such as FileNewL . These methods may leave with...
Decoding Thumbnails
The CImageDecoder class also has support for decoding image thumbnails. This support is fairly generic and even decoders for image formats that don't have a thumbnail concept may implement it by generating thumbnails from a normal image on request. However simple the API is, this feature is a bit tricky to use. CImage-Decoder SetImageTypeL can be used to switch between image types. You should use EImageTypeThumbnail for the thumbnail and EImageTypeMain for the main image. It is possible to call...
Recording to a Named File
We are going to look at code that does something similar to the audio input-output stream example it records something and then plays it back. For this API, we record into a file and the file remains afterwards. Skipping the declaration, the construction is fairly predictable. The key iUtility CMdaAudioRecorderUtility void CRecordAudio RecordL const TDesC amp r This requests the opening of a file. We'll return later to some of the subtleties of this call. The key is that it is asynchronous and...
Use RFile and TMMSource Instead of Passing a File Name
When using CMdaAudioPlayerUtility, there are a number of ways of stating which file to play. These include static CMdaAudioPlayerUtility NewFilePlayerL const TDesC amp aFileName, MMdaAudioPlayerCallback amp aCallback, TInt aPriority EMdaPriorityNormal, TMdaPriorityPreference aPref static CMdaAudioPlayerUtility NewL MMdaAudioPlayerCallback amp aCallback, TInt aPriority EMdaPriorityNormal, TMdaPriorityPreference aPref The question is which sequence to use The reason for the large number of...
Symbian Press Books
Further information about Symbian Press books can be found at devel-oper.symbian.com books. If you are new to C development on Symbian OS and wish to find out more about the idioms, toolchain and application framework, we recommend these books to get you started Developing Software for Symbian OS 2nd Edition, Babin. John Wiley amp Sons. Quick Recipes on Symbian OS Mastering C Smartphone Development, Aubert etal. John Wiley amp Sons. The following book is a more detailed reference book for...
Use CMdaAudioPlayerUtility to Play Tone Sequence Files
At face value, the way to play a tone sequence file is to use CMda-AudioToneUtility. However, it is also possible to play such files using CMdaAudioPlayerUtility -device configurations include controllers to play these files, so you can play them like any other audio file. In general, using CMdaAudioPlayerUtility is preferred to CMda-AudioToneUtility. Although it uses more runtime resources - an extra thread and several plug-ins to support controllers - the same CMdaAudioPlayerUtility code can...
Tone Utility
This section is about playing tones-simple tones, DTMF and tone sequences. This functionality is provided by CMdaAudioToneUtil-ity.9 Although the same class is used, the calling sequences differ and can be dealt with separately. CMdaAudioToneUtility shares many characteristics with CMda-AudioOutputStream- it acts as a relatively thin layer over DevSound and neither runs in a separate thread nor uses a controller object. Nevertheless there are differences rather than supplying data in buffers,...
Selecting the Image Format
The formats that are supported are determined by TCameraInfo iImageFormatsSupported. This is a bitfield of values from the enumeration CCamera TFormat. When using cameras that support a number of formats, we need to choose the format most suitable for our application. CCamera TFormat iFormat TInt iSizeIndex Initial camera querying to get supported capture sizes iSizeArray is defined elsewhere as RArray lt TSize gt for TInt i 0 i lt iInfo.iNumImageSizesSupported i i, iFormat SelectBestSize looks...
The ECOM Framework
The ECOM framework is the Symbian OS plug-in framework. It is intended to be synonymous with Microsoft's Component Object Model COM architecture, with the first letter coming from the original name of Symbian OS - EPOC32 - by convention with other Symbian components, such as ESock, ETel and EKern. A plug-in is a way to extend the functionality of an application by allowing different implementations of a defined interface to be provided. Many Symbian OS system services use ECOM plug-ins for...
Variations 1
Based on these examples, there are quite a few variations you can implement Display the duration of the clip and track progress. The best way to get the duration is by the following call TMMFDurationInfo Duration TTimeIntervalMicroSeconds amp aDuration This is better than the alternatives as it also returns a TMMFDurationInfo value that can indicate if the duration is 'valid' so the returned aDuration6 is important , is unknown for some reason or is 'infinite' - the latter makes more sense for...
OpenSL ES
OpenSL ES is another standard from the Khronos Group. It is the sister standard of OpenGL ES in the audio area. It has been designed to minimize fragmentation of audio APIs between proprietary implementations and to provide a standard way to access audio hardware acceleration for application developers. OpenSL ES is also a royalty-free open API and is portable between platforms, like OpenMAX. In fact, OpenMAX AL and OpenSL ES overlap in their support for audio playback, recording and MIDI...
Dont Use CMdaAudioOutputStream for Network Streaming
It is a common misconception, which is understandable given the name of the class, however, CMdaAudioOutputStream is not suitable for streamed data, in the sense of streaming audio over a network. The name of the class goes back to the days of the Media Server on Symbian OS v7.0 see Section 1.5.1 . Its reasoning is largely lost now, but its intended use case was for playing unwrapped, raw-encoded data, to be fed into a DevSound codec. If you receive this sort of data over the network, this may...
Accessing the Camera
The ECam API is an open, extensible, generic API that provides functionality to control the digital camera on the phone and to request and receive specific image data from it. Figure 3.1 shows where the ECam API is placed in the multimedia subsystem and the other multimedia components it interacts with. Further information about the Multimedia Framework can be found in Chapters 4 and 5 of this book the Image Conversion Library is discussed in Chapter 6. The onboard camera component provides...
Controlling the Audio Output Audio Resource Control
In the CVideoPlayerUtility NewL method we saw there were two parameters aPriority and aPref. These are used to control access to an audio resource such as a loudspeaker. The parameter, aPriority, is used in the situation where two clients require access to the same sound device such as the loudspeaker. While a video is being played, its audio track uses the speaker. If, during the video playback, the calendar client wishes to play an audible tone to indicate an upcoming appointment, the...
Use APPARC to Recognize Audio and Video
As we noted in Chapters 4 and 5, the multimedia framework provides separate APIs for audio and video clip usage. Depending on the applications you are developing, you may want to handle both - i.e. given a file, you may want to use the video APIs for video files and the audio APIs for audio files. The question is how do you do this One approach would be to guess to try to open a file as video and then open it as audio if this fails. This should be avoided for several reasons USE APPARC TO...
Multimedia Architecture
The current multimedia architecture was introduced in Symbian OS v7.0s and has been extended and improved in subsequent releases. This chapter provides an overview of the architecture and its underlying functionality, followed by a description of the separate parts of the multimedia subsystem and the associated multimedia APIs available. Among other things, the Symbian OS multimedia APIs can be used to play and record audio and video data perform image manipulation and processing access the...
Controlling the Video that Is Recorded
The frame rate for the recording can be specified using the SetVideo-FrameRateL method. You can read the current frame rate being used with the VideoFrameRateL method. void SetVideoFrameRateL TReal32 aFrameRate TReal32 VideoFrameRateL const The frame rate is a floating-point number specifying the number of frames per second. From Symbian OS v9.4 onwards, you can specify whether the frame rate is fixed or variable using the SetVideoFrameRateFixedL method. void SetVideoFrameRateFixedL TBool...
Displaying Images
The CImageDisplay API can sometimes be seen as an analog of the CImageDecoder API however it was designed to allow an application to easily display images, especially animated multiframe ones. The CImageDisplay class8 is not a replacement for CImage-Decoder, as it does not allow much access to the underlying image. It abstracts an application from the image details and, where it is possible to do so, from the plug-in capabilities as well. This API shares many design elements with...
Getting Video Information
The frame rate describes the number of images in the video per unit of time. For instance, a frame rate of 15 frames per second fps indicates that the video contains 15 separate images for each second of that video. For videos on mobile phones, frame rates ranging from 8fps up to 30fps are typical. For reference, a modern console video game would run at 30-60fps and a video telephony call would have a frame rate of 10-15 fps. To get the frame rate of the currently open video, use the...
Accessing Metadata
A video may contain extra items of information such as the video title or author. This information is held in a form called metadata. The metadata format depends on the type of video being played. If the controller supports it, it is possible to read these items of metadata from the video file. You can read the number of metadata entries using the Number-OfMetaDataEntriesL method TInt NumberOfMetaDataEntriesL const This tells you the maximum number of metadata items you can read from the file....
Decoding to a Different Size
Let us consider a rather common use case of decoding an image to a different usually reduced size from its original. That use case can often be seen in applications that do not control the way images are fed into an application, such as image-viewing applications, applications that can exchange images, MMS applications, etc. We would like to be able to scale images to a certain size, either so it is possible to fit several image thumbnails into the application workspace or to show a small image...
Exif Metadata Processing
Exif metadata processing is closely coupled with the CImageDecoder API,7 which means that an image decoder object has to be instantiated in order to gain access to the metadata information. 7Nokia had created an API for Exif metadata processing before Exif support was introduced in Symbian OS v9.1. Nokia-owned APIs are not part of the core Symbian OS SDK. All the Symbian-provided Exif APIs are excluded from the S60 3rd Edition SDKs. All the UIQ 3 SDKs use core Symbian Exif APIs. Exif metadata...
DevSound 1
DevSound properly CMMFDevSound is the fundamental component at the bottom of the MMF API stack. It encapsulates the audio adaptations for devices. Under DevSound are found the lower-level audio codecs, the audio policy what happens if more than one client requests to play or record concurrently and the links to the various audio devices. These features are common to all the DevSound implementations on different Symbian smartphones, but the underlying architecture varies from one manufacturer to...
Using the CImageEncoder API
The CImageEncoder API shares many usage patterns and data structures with the CImageDecoder API. Its basic usage scenario is essentially the same 1. An encoder object is to be created by using one of the synchronous factory methods CImageEncoder DataNewL or CImage-Encoder FileNewL . 2. Some optional encoding parameters may be adjusted synchronously. 3. An asynchronous encoding process is initiated by calling CImage-Encoder Convert . 4. The application waits for the request to be accomplished...
Miscellaneous APIs
There are several helper classes which can be used by an application to perform pixel-level operations on native bitmaps see Figure 6.12 . These classes were designed as re-usable components for decoder plug-ins as they do a fair amount of pixel operations , however since they are published APIs, they can be used by third-party applications as well. NewImageProcessorL CFbsBitmap amp , TSize amp , TDisplayMode, TBool CimageProcessor NewimageProcessorL CFbsBitmap amp , Tint, TDisplayMode, TBool...
Streamed Decoding
The image decoder component has basic support for streamed decoding. It is possible to decode an image in which physical content is not available immediately in full. This kind of use case is mostly for web-browser-like image downloading. The image decoder API does not support 'true' streamed decoding, which means that an application has to accumulate incoming image data on its own. Figure 6.7 shows an activity diagram for the following streamed decoding scenario. If CImageDecoder NewL leaves...
Plugin Discovery
The CImageDecoder class has some functionality which allows retrieval of information about the decoder plug-in being used. These informational methods can be useful for debugging or for implementing plug-in specific behavior which is, in general, not very good practice . The following method allows you to get the UID that uniquely identifies the plug-in In fact there could be several plug-ins each one with its own unique UID able to handle the same image type UID. It is the framework's job to...
Creating the Playback Classes
Both CVideoPlayerUtility and CVideoPlayerUtility2 are instantiated by calling their static NewL methods see Figure 4.2 static CVideoPlayerUtility NewL MVideoPlayerUtilityObserver amp aObserver, Tint aPriority, TMdaPriorityPreference aPref, RWsSession amp aWs, CWsScreenDevice amp aScreenDevice, RWindowBase amp aWindow, const TRect amp aScreenRect, const TRect amp aClipRect static CVideoPlayerUtility2 NewL MVideoPlayerUtilityObserver amp aObserver, Tint apriority, TMdaPriorityPreference aPref The...
Opening a Video
There are three methods that are available for opening a file containing a video You can specify a file name, a handle to an already opened file, or a TMMSource, which allows access to a DRM-protected source. There is one method that allows you to open a video contained in a descriptor buffer void OpenDesL const TDesC8 amp aDescriptor, TUid aControllerUid KNullUid There is one method that allows you to open a video for streaming void OpenUrlL const TDesC amp aUrl, TInt aIapId KUseDefaultIap,...
Setting up Encoding Parameters
Apart from the instantiation-time parameters, there are parameters that can be set up before the encoding process takes place. This kind of parameter setup split is used to keep the CImageEncoder interface as generic as possible while allowing format-specific settings to be supplied by an API client if necessary. The setup mechanism for CImageEncoder that is specific to each plug-in is driven by parameter data rather than custom interface functions see Figure 6.8 for a UML diagram There are two...
Client API Introduction
For playback, there are two classes that contain all of the APIs that can be used, CVideoPlayerUtility found in videoplayer.h and CVideoPlayerUtility2 found in videoplayer2.h . For either class, you must link against mediaclientvideo.lib. CVideoPlayerUtility can be used with any video controller, whereas CVideoPlayerUtility2 is only available from Symbian OS v9.5 onwards and can only be used with video controllers that support graphics surfaces see Section 4.6.3 for more information about...
Transforming Images
The image transformation API9 is designed to aid applications to transform images which are encoded in a format other than native bitmap format. Native bitmaps can be efficiently transformed using the bitmap 9This API is not included in the S60 3rd Edition SDKs. transformation API see Section 6.6 . The image transformation API is an extendable, plug-in-based framework, which shares many design principles with CImageDecoder. At the time of writing, CImageTransform was able to perform the...