Issue with Bluetooth audio and CameraKit
Dear Snap Team,
We believe there is a potential issue with using CameraKit for video recording when a Bluetooth headset is connected to an iPhone.
Precondition
Before starting a CameraKit session, we:
- configure the shared AVAudioSession instance to allow Bluetooth audio (HFP profile to enable capturing audio from the headset mic)
let session = AVAudioSession.sharedInstance() try session.setCategory( AVAudioSession.Category.playAndRecord, mode: AVAudioSession.Mode.default, options: [.allowBluetooth]) try session.setActive(true, options: .notifyOthersOnDeactivation)
- configure an AVCaptureSession as follows:
let captureSession = AVCaptureSession() captureSession.sessionPreset = .hd1920x1080 captureSession.usesApplicationAudioSession = true captureSession.automaticallyConfiguresApplicationAudioSession = false let audioDevice = AVCaptureDevice.default(for: .audio) do { let audioInput = try AVCaptureDeviceInput(device: audioDevice!) if captureSession.canAddInput(audioInput) { captureSession.addInput(audioInput) } else { print("Could not add audio input to session") } } catch { print("Could not create audio input") }
Initialise an AVSessionInput instance as
let input = AVSessionInput(captureSession, true)
Expected ResultCameraKit uses an audio device from the preconfigured AVAudioSession and AVCaptureSession when recording a video (i.e. Bluetooth headset mic when AirPods are connected to the iPhone)
- CameraKit does not make any changes to AVAudioSession
Once video recording is finished, the user can resume audio playback to their Bluetooth headset from Apple Music / Spotify
Actual ResultCameraKit ignores any configuration settings previously made to AVAudioSession and re-initialises it with
options: [.defaultToSpeaker]
- CameraKit uses the built-in iPhone mic to record audio
- Audio playback to the Bluetooth headset cannot be resumed
Comments
-
Hey @Sani ! Thanks for posting on the forum as always, and apologies for this issue you're experiencing. Confirming I've created an internal ticket for our eng team to look at. Fyi, seems like there's another developer also experiencing a somewhat related issue with audio here: https://community.snap.com/snapar/discussion/2403/camerakit-crashes-if-on-audio-call-ios#latest
I'll get back to you as soon as I hear back.
0 -
Hi @stevenxu hope you are doing well and thanks for a super prompt feedback. The details are as follows:
Device models:
iPhone 12 Pro Max iOS (16.3.1)
iPhone X iOS (14.6)
iPhone 11 iOS 16.4.1
iPhone XR iOS 16.0.2
2. Camera Kit SDK version 1.20.0We are happy to do more testing on more devices as well, please let me know.
Thank you 🙌
P.S. - we have done some sanity check and tested other partner apps using the Camera Kit and they all seem to have the same/similar issues.
0 -
Hi @Sani, thank you for reporting this issue. I am able to reproduce it. It looks like Camera Kit SDK is overriding the
AVAudioSession
category
downstream from where you set it.I have opened a ticket so that the team can investigate.
In the meantime, since Camera Kit is changing the category, a potential workaround is to set the category every time the lens is applied, for example, in the completion of
CameraController.applyLens(_:completion)
. I appreciate this is not ideal but thought I would mention it as something to try. Thanks again.1