Issue with Bluetooth audio and CameraKit

Options
Sani
Sani Posts: 13 🔥

Dear Snap Team,

We believe there is a potential issue with using CameraKit for video recording when a Bluetooth headset is connected to an iPhone.

Precondition
Before starting a CameraKit session, we:

  • configure the shared AVAudioSession instance to allow Bluetooth audio (HFP profile to enable capturing audio from the headset mic)
let session = AVAudioSession.sharedInstance()
try session.setCategory(
        AVAudioSession.Category.playAndRecord,
        mode: AVAudioSession.Mode.default,
         options: [.allowBluetooth])
try session.setActive(true, options: .notifyOthersOnDeactivation)
  • configure an AVCaptureSession as follows:
let captureSession = AVCaptureSession()
captureSession.sessionPreset = .hd1920x1080
captureSession.usesApplicationAudioSession = true
captureSession.automaticallyConfiguresApplicationAudioSession = false

let audioDevice = AVCaptureDevice.default(for: .audio)
do {
  let audioInput = try AVCaptureDeviceInput(device: audioDevice!)
  if captureSession.canAddInput(audioInput) {
    captureSession.addInput(audioInput)
  } else {
    print("Could not add audio input to session")
  }
} catch {
  print("Could not create audio input")
}
  • Initialise an AVSessionInput instance as
    let input = AVSessionInput(captureSession, true)
    Expected Result

  • CameraKit uses an audio device from the preconfigured AVAudioSession and AVCaptureSession when recording a video (i.e. Bluetooth headset mic when AirPods are connected to the iPhone)

  • CameraKit does not make any changes to AVAudioSession
  • Once video recording is finished, the user can resume audio playback to their Bluetooth headset from Apple Music / Spotify
    Actual Result

  • CameraKit ignores any configuration settings previously made to AVAudioSession and re-initialises it with options: [.defaultToSpeaker]

  • CameraKit uses the built-in iPhone mic to record audio
  • Audio playback to the Bluetooth headset cannot be resumed

Comments

  • stevenxu
    stevenxu Posts: 611 👻
    Options

    Hey @Sani ! Thanks for posting on the forum as always, and apologies for this issue you're experiencing. Confirming I've created an internal ticket for our eng team to look at. Fyi, seems like there's another developer also experiencing a somewhat related issue with audio here: https://community.snap.com/snapar/discussion/2403/camerakit-crashes-if-on-audio-call-ios#latest

    I'll get back to you as soon as I hear back.

  • stevenxu
    stevenxu Posts: 611 👻
    Options

    @Sani could you also provide the following:

    • Device model(s)
    • CameraKit SDK version
  • Sani
    Sani Posts: 13 🔥
    Options

    Hi @stevenxu hope you are doing well and thanks for a super prompt feedback. The details are as follows:

    Device models:
    iPhone 12 Pro Max iOS (16.3.1)
    iPhone X iOS (14.6)
    iPhone 11 iOS 16.4.1
    iPhone XR iOS 16.0.2
    2. Camera Kit SDK version 1.20.0

    We are happy to do more testing on more devices as well, please let me know.

    Thank you 🙌

    P.S. - we have done some sanity check and tested other partner apps using the Camera Kit and they all seem to have the same/similar issues.

  • stevenxu
    stevenxu Posts: 611 👻
    Options

    @Sani Thank you! I've updated the eng ticket with your latest info. Feel free to follow up .. next Tuesday if you don't hear back. Our team is currently busy with a few items so appreciate your patience!

  • arashp
    arashp Posts: 52 🔥🔥
    Options

    Hi @Sani, thank you for reporting this issue. I am able to reproduce it. It looks like Camera Kit SDK is overriding the AVAudioSession category downstream from where you set it.

    I have opened a ticket so that the team can investigate.

    In the meantime, since Camera Kit is changing the category, a potential workaround is to set the category every time the lens is applied, for example, in the completion of CameraController.applyLens(_:completion). I appreciate this is not ideal but thought I would mention it as something to try. Thanks again.

  • rockib13
    rockib13 Posts: 1
    Options

    @stevenxu the workaround mentioned above did not work for my app. Wondering if there have been any updates on this issue?