Skip to content

build-aux: Replace Flatpak modules with pre-compiled dependencies #10934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

build-aux: Replace Flatpak modules with pre-compiled dependencies #10934

wants to merge 2 commits into from

Conversation

tytan652
Copy link
Collaborator

@tytan652 tytan652 commented Jun 30, 2024 •
edited
Loading

Description

Important

This PR relies on https://github.com/tytan652/obs-deps-bst, the PR is also meant to discuss including the BuildStream-based project under the obsproject org.

This PR is x86_64 only, if it gets approved, #9979 will be refactored with aarch64 tarballs and I will try to work on documenting obs-deps-bst.

To reduce compile time and prepare for aarch64 support, dependencies compilation except CEF is moved to a BuildStream project junctionned with Freedesktop SDK.

The BuildStream project is configured to build dependencies in a Flatpak-like environment.

Motivation and Context

Move away dependency compile time, in an aarch64 context it moves away 6 hours of build time to another repo with ~3 hours of build time* without cache.
So only the CEF wrapper and OBS have to be built on this repo CI, so ~1h IIRC for aarch64 without cache.

*Build time is reduced since BuildStream tries to build elements in parallel when possible even with QEMU emulation.

Also on obs-deps-bst side, sources are cached so outage of independent git remote is not an issue if cached.

How Has This Been Tested?

Types of changes

  • Tweak (non-breaking change to improve existing functionality)
  • Performance enhancement (non-breaking change which improves efficiency)

Checklist:

  • My code has been run through clang-format.
  • I have read the contributing document.
  • My code is not on the master branch.
  • The code has been tested.
  • All commit messages are properly formatted and commits squashed where appropriate.
  • I have included updates to all appropriate documentation.

@tytan652 tytan652 added Linux Affects Linux CI Seeking Testers Build artifacts on CI labels Jun 30, 2024
@tytan652 tytan652 mentioned this pull request Jun 30, 2024
Draft
6 tasks
@tytan652 tytan652 force-pushed the fp_obs-deps-bst branch 3 times, most recently from 4f766ad to f94c313 Compare February 7, 2025 19:40
@tytan652 tytan652 marked this pull request as ready for review February 12, 2025 09:54
@tytan652 tytan652 force-pushed the fp_obs-deps-bst branch 4 times, most recently from 2b5208b to aa6a037 Compare May 8, 2025 12:33
@tytan652 tytan652 added this to the OBS Studio 32.0 milestone Jun 16, 2025
@RytoEX RytoEX requested a review from GeorgesStavracas June 16, 2025 21:25
@tytan652 tytan652 force-pushed the fp_obs-deps-bst branch 4 times, most recently from fcdf081 to 5de190d Compare July 18, 2025 05:46
@RytoEX
Copy link
Member

RytoEX commented Aug 6, 2025

@GeorgesStavracas Thoughts on this approach?

GeorgesStavracas
GeorgesStavracas approved these changes Aug 6, 2025
View reviewed changes
Copy link
Member

@GeorgesStavracas GeorgesStavracas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I concur with this approach, with 2 conditions:

  • Since we're very close to FDO 25.08, might be better to use it when it's released? Thoughts?
  • The obs-deps-bst repository has to be moved to the obsproject group first

(Interestingly, last week I talked to the new Fedora project leader Jef Spaleta and we agreed that moving to FDO-SDK and vendoring our dependencies is a decision that the Fedora project supports)

@tytan652
Copy link
Collaborator Author

tytan652 commented Aug 6, 2025 •
edited
Loading

  • Since we're very close to FDO 25.08, might be better to use it when it's released? Thoughts?

I think certainty on when 25.08 releases before when OBS Studio next beta is needed to be able to do the switch.

Edit: If we want to try to switch to Freedesktop SDK 25.08, we need:

  • 25.08 being out in stable before OBS 32 beta
  • flathub-infra action-images providing an image with 25.08 pre-installed

@PatTheMav
Copy link
Member

The changes in this PR look fine to me. Would it be possible to integrate the BuildStream code into obs-deps itself, bringing the actions over and adding the Linux-specific workflow jobs? The BuildStream-specific files could easily all live in its own deps.flatpak directory and we'd have all our dependency-building code in one place (and thus would also have all PRs and issues associated with them in a single place) which would help with visibility.

@tytan652
Copy link
Collaborator Author

tytan652 commented Aug 7, 2025 via email

@PatTheMav
Copy link
Member

Because of GHA caching limits, the Buildstream project needs it's own repo. Le 7 août 2025 22:00:37 GMT+02:00, Patrick Heyer @.> a écrit :
…
PatTheMav left a comment (obsproject/obs-studio#10934) The changes in this PR look fine to me. Would it be possible to integrate the BuildStream code into obs-deps itself, bringing the actions over and adding the Linux-specific workflow jobs? The BuildStream-specific files could easily all live in its own deps.flatpak directory and we'd have all our dependency-building code in one place (and thus would also have all PRs and issues associated with them in a single place) which would help with visibility. -- Reply to this email directly or view it on GitHub: #10934 (comment) You are receiving this because you authored the thread. Message ID: @.>

Got it. I think I can give it a simple review pass for any obvious gotchas, particularly with an eye towards project data etc., but otherwise I'd probably defer to you and @GeorgesStavracas when it comes to the repo itself, so you can maintain it as independently as obs-deps is maintained (so no expectations for stuff to work exactly the same way, etc.).

@PatTheMav
Copy link
Member

Gave the buildstream repo a quick glance, seems to be fine, couldn't spot any (ab)use of secrets or GitHub tokens (not that I expected any).

Personally I'd probable prefer to call it obs-deps-buildstream (if "BuildStream" is more or less exclusively associated with Flatpak that makes it superfluous to mention it in the repo name), but I'll leave this to @RytoEX to pick up.

@tytan652
Copy link
Collaborator Author

tytan652 commented Aug 9, 2025 •
edited
Loading

"bst" is a common abbreviation but going for "buildstream" is fine too (project.conf would need an update).

if "BuildStream" is more or less exclusively associated with Flatpak that makes it superfluous to mention it in the repo name

It isn't, it can be used to build OCI images, entire Linux distribution and in our case dependencies for our Flatpak (even stuff on macOS or WSL).
This is not for the short-term, but I has some thought about making an OCI image from it for static analysis if judged helpful.

@RytoEX RytoEX self-assigned this Aug 22, 2025
@tytan652 tytan652 mentioned this pull request Aug 27, 2025
Open
@tytan652
build-aux: Replace Flatpak modules with pre-compiled dependencies
46ff60e 
To reduce compile time and prepare for aarch64 support, dependencies
compilation except CEF is moved to a BuildStream project junctionned
with Freedesktop SDK.

The BuildStream project is configured to build dependencies in a
Flatpak-like environment.
@tytan652
build-aux,CI: Replace Flatpak KDE SDK with Freedesktop SDK
df6320c 
The Qt components are from the same repo as other dependencies.

The KDE Platform was never a good fit for OBS Studio for multiple
reasons:
- Life cycle too short and misaligned with OBS Studio causing users to
  get EOL notification too often.
- Third-party Qt plugins were allowed which could change Qt's behavior.
- KDE exclusives sandboxed holes were inherited.
@tytan652
Copy link
Collaborator Author

tytan652 commented Aug 31, 2025 •
edited
Loading

Due to libobs now depending on SIMDe, the devel tarball is no longer cleaned up (devtools still is).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@GeorgesStavracas GeorgesStavracas GeorgesStavracas approved these changes

Assignees

@RytoEX RytoEX

Labels
CI Linux Affects Linux Seeking Testers Build artifacts on CI
Projects
OBS Studio 32.0 PR Considerations
Status: Backlog
Milestone
OBS Studio 32.0
Development

Successfully merging this pull request may close these issues.
4 participants
@tytan652 @RytoEX @PatTheMav @GeorgesStavracas