Skip to content

Add support for publishing in background mode: VideoToolBox now supports background mode #626

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
themaxsmith opened this issue Dec 12, 2019 · 3 comments
Closed

Add support for publishing in background mode: VideoToolBox now supports background mode #626

themaxsmith opened this issue Dec 12, 2019 · 3 comments
Labels
enhancement Indicates that an issue is feature request.

Comments

@themaxsmith
Copy link

Is your feature request related to a problem? Please describe.

Hi, I am moving my app (SeasonCast) from a custom modified version of LFLiveKit to HashinKit.swift due to it not being updated in 3 years and it's memory leak issues. I would like to have support for streaming while the app is in background mode (I know in the past this was not possible due to VideoToolBox but in recent versions of iOS it works). I got VideoToolBox library to work in the background of LFLiveKit by enabling audio in the background and a minor modification to its equivalence of the H264Encoder.swift class by removing its background observer (https://github.com/LaiFengiOS/LFLiveKit/blob/master/LFLiveKit/coder/LFHardwareVideoEncoder.m). I think it would be possible to do the same thing for this library since it's laid out in a similar matter. I have attempted to modify the code but it seems that the problem preventing it from working in background mode is before H264Encoder.swift and I can't seem to find the reason why.

Describe the solution you'd like

I would like the framework to behave in the same exact manner when background mode - audio is not enabled.

When the app has background mode - audio enabled. It should show the microphone recording icon when in background mode (because the microphone would still be active) and if the user is publishing the microphone should still be streaming but the camera should be muted (last frame).
@themaxsmith themaxsmith changed the title Add support for h.264 encoding background mode: VideoToolBox now supports background mode Add support for publishing in background mode: VideoToolBox now supports background mode Dec 12, 2019
@shogo4405
Copy link
Collaborator

shogo4405 commented Feb 9, 2020 •
edited
Loading

@themaxsmith

VideoToolBox now supports background mode

Can I see document?

@shogo4405 shogo4405 modified the milestone: 1.0.4 Feb 9, 2020
@troy-lamerton
Copy link
Contributor

In background, frames were not encoded due to error:
Calls to VTCompressionSessionEncodeFrame are returning kVTInvalidSessionErr.
To workaround, compression session must be invalidated when the app enters background (detect with ios notification).

  1. Use background task api to let code keep running
    https://developer.apple.com/documentation/uikit/uiapplication/1623031-beginbackgroundtask
  2. Invalidate compression session when app enters background, like this:
// invalidate compression session
let realWidth = videoSettings[.width]
videoSettings[.width] = 1
videoSettings[.width] = realWidth
  1. Now you can give CVPixelBuffer to the input method and it will be encoded, stream stays online.

This works for the duration of background task api, which is 30-180 seconds. Would be interested in possible way to continue session for longer in background.

@shogo4405
Copy link
Collaborator

Thank you feedback. Currently, priority is low. I will close.
If I feel like it, reopen then implement.

@Rikukuku Rikukuku mentioned this issue Oct 28, 2021
Closed
@philcluff philcluff mentioned this issue Jan 13, 2022
Open
@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 2, 2024
@shogo4405 shogo4405 added the enhancement Indicates that an issue is feature request. label Jan 17, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
enhancement Indicates that an issue is feature request.
Milestone
No milestone
Development

No branches or pull requests
3 participants
@shogo4405 @troy-lamerton @themaxsmith