Media Player Queue Enhancement

Overview

I’ve enhanced the AndroidMediaPlayer to support queuing multiple audio tracks with playback callbacks, specifically designed for demos using AWS Polly-generated MP3s and speech marks.

Files Created

1. shared/src/commonMain/kotlin/io/github/bsautner/krill/media/AudioTrack.kt
   • Data models: AudioTrack, SpeechMark, TrackInfo
   • Interface: TrackPlaybackListener for playback callbacks
2. shared/src/commonMain/kotlin/io/github/bsautner/krill/media/NarrationDemo.kt
   • Example code showing how to use the new queue functionality
   • Demonstrates both simple and advanced usage patterns
3. docs/MEDIA_PLAYER_QUEUE.md
   • Comprehensive documentation
   • Usage examples
   • API reference
   • Implementation details

Files Modified

1. shared/src/commonMain/kotlin/io/github/bsautner/krill/Platform.kt
   • Updated MediaPlayer interface to add:
     • suspend fun playQueue(tracks: List<AudioTrack>, listener: TrackPlaybackListener?)
     • fun stop()
     • fun clearQueue()
2. shared/src/androidMain/kotlin/io/github/bsautner/krill/media/AndroidMediaPlayer.kt
   • Implemented the new queue functionality
   • Added track transition detection via ExoPlayer listeners
   • Fetches and parses marks.json files
   • Calculates duration from speech marks
   • Provides callbacks for track start, complete, and errors

Key Features

✅ Queue Support: Add multiple MP3s with their marks.json URLs
✅ Track Callbacks: Get notified when each track starts and completes
✅ Duration Info: Automatically calculated from speech marks
✅ Speech Marks Access: Full access to word and sentence timing data
✅ Error Handling: Graceful error handling with callbacks
✅ Simple API: Can be used with or without listener

Usage Example

suspend fun playDemo() {
    val tracks = listOf(
        AudioTrack(
            mp3Url = "https://cdn.example.com/intro.mp3",
            marksJsonUrl = "https://cdn.example.com/intro.marks.json"
        ),
        AudioTrack(
            mp3Url = "https://cdn.example.com/features.mp3",
            marksJsonUrl = "https://cdn.example.com/features.marks.json"
        )
    )
    
    val listener = object : TrackPlaybackListener {
        override fun onTrackStarted(trackIndex: Int, trackInfo: TrackInfo) {
            Logger.d("Track $trackIndex started - ${trackInfo.durationMs}ms")
        }
        
        override fun onTrackCompleted(trackIndex: Int) {
            Logger.d("Track $trackIndex done")
        }
        
        override fun onQueueCompleted() {
            Logger.d("All done!")
        }
        
        override fun onError(trackIndex: Int, error: String) {
            Logger.d("Error: $error")
        }
    }
    
    mediaPlayer.playQueue(tracks, listener)
}

How It Works

sequenceDiagram
    participant App
    participant MediaPlayer
    participant ExoPlayer
    participant CDN
    
    App->>MediaPlayer: playQueue(tracks, listener)
    MediaPlayer->>CDN: Fetch marks.json files (parallel)
    CDN-->>MediaPlayer: Speech marks data
    MediaPlayer->>MediaPlayer: Calculate durations
    MediaPlayer->>ExoPlayer: Queue all MP3s
    ExoPlayer->>ExoPlayer: Start playback
    
    loop For each track
        ExoPlayer->>MediaPlayer: onPlaybackStateChanged(STATE_READY)
        MediaPlayer->>App: onTrackStarted(index, trackInfo)
        ExoPlayer->>MediaPlayer: onMediaItemTransition
        MediaPlayer->>App: onTrackCompleted(index)
    end
    
    ExoPlayer->>MediaPlayer: onPlaybackStateChanged(STATE_ENDED)
    MediaPlayer->>App: onQueueCompleted()

Architecture

1. Queue Setup: Call playQueue() with a list of AudioTrack objects
2. Marks Fetching: The player fetches all marks.json files in parallel
3. Duration Calculation: Duration is calculated from the last mark’s timestamp
4. Playback: ExoPlayer plays all tracks in sequence
5. Callbacks:
   • onTrackStarted when each track begins (includes TrackInfo with marks and duration)
   • onTrackCompleted when each track finishes
   • onQueueCompleted when all tracks are done
   • onError if any errors occur

Next Steps

To use this in your demo:

1. Upload your Polly-generated MP3s and marks.json files to a CDN or server
2. Create AudioTrack objects with the URLs
3. Implement a TrackPlaybackListener to handle callbacks
4. Use speech marks for synchronized text display or animations