Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug: 2.4.x not compiling with ffmpeg 7 #1578

Open
Behinder opened this issue Oct 5, 2024 · 23 comments
Open

Bug: 2.4.x not compiling with ffmpeg 7 #1578

Behinder opened this issue Oct 5, 2024 · 23 comments
Labels
type/bug

Comments

@Behinder
Copy link

Behinder commented Oct 5, 2024

Observed Behavior

error during make
wyn.txt

Expected behaviour

creating binary

Steps to reproduce

  1. Install lot of dependencies - cmake doest not suggest what to install so the BUILDING.md is incomplete
  2. cmake ../src
  3. make -j8

Environment

Tested on Linux Mint and Ubuntu latest - arm64 and MacOS Apple Silicon inside ubuntu in docker container
@Julusian
Copy link
Member

Julusian commented Oct 7, 2024

It looks like your system is using ffmpeg 7 which introduced some breaking changes.
This is fixed in master, which I suppose would be reasonable to backport to 2.4.1

I should point out that we generally only recommend using the LTS versions of ubuntu, those are what we test with.

Install lot of dependencies - cmake doest not suggest what to install so the BUILDING.md is incomplete

What did you need to install that wasn't documented?

@Behinder
Copy link
Author

Behinder commented Oct 7, 2024 via email

@Julusian
Copy link
Member

Julusian commented Oct 7, 2024

Strange, we are running builds in ubuntu 22.04 and 24.04 in github actions, using solely system dependencies to ensure that it remains building successfully on those. As well as one on ubuntu 22.04 which uses our packaged dependencies
The script we use for the system one is

server/.github/workflows/linux-system.yml

Lines 23 to 48 in d37b9d0

- name: Setup environment
run: |
sudo apt-get update
sudo ./tools/linux/install-dependencies
sudo apt-get -y install software-properties-common
sudo add-apt-repository ppa:casparcg/ppa
sudo apt-get -y install libboost-all-dev \
libavcodec-dev \
libavformat-dev \
libavdevice-dev \
libavutil-dev \
libavfilter-dev \
libswscale-dev \
libpostproc-dev \
libswresample-dev \
casparcg-cef-117-dev
- name: Run build
run: |
mkdir build
cd build
cmake ../src -DUSE_STATIC_BOOST=OFF -DUSE_SYSTEM_FFMPEG=ON -DUSE_SYSTEM_CEF=ON
make -j2

and you can see a successful run from it last week https://github.com/CasparCG/server/actions/runs/11076901193/job/30780996777
so it should work locally too.
I can't speak for mint though.

Haven’t checked that on Linux but on MacOS ffmpeg 6 so probably also 7
installs his corresponding headers in /opt/local/libexec/ffmpeg6/include
instead of /opt/local/include - I don’t know the reason but it must taken
into considerations when using as dependency.

Both of those paths are rather non standard for linux, so they sound like a macos 'quirk', that as we don't support macos don't handle currently. I don't have a problem with supporting other paths, but don't know where that should fit into the cmake config

I cannot tell you at the moment what was missing , I would have to repeat
whole process. Whenever I did make command there was always something
missing like even OpenGL, GLX, SFMTL.

Strange, I can't explain this assuming you were using ubuntu/mint there.

Also my concern is that your script downloading code that uses boost
library but the code is copyrighted in 2001!

The version we use in 2.4 is from 2018, but it is likely that some of the copyright notices haven't been updated.
For reference, I am intentionally using an older version, matching the oldest version of ubuntu that we want to support. The intention is to minimise the risk of using 'new' features that won't be available on the older ubuntu versions. Tbh this is probably overly cautious, but also there hasn't been a reason to update it. It is newer in master, matching the version used in ubuntu 22.04

I think 2 years ago I wanted macOS version and only one person wanted to
help me with this, sad. So I am going to work on CasparCg using MGL project
or to create it just in Metal. Unless you do mind because licensing.

Personally, I don't mind the idea of having some code in this repository to make it buildable/usable on macos. But I am hesitant to accept an implementation using metal.
I agree with the prior comments that a macos implementation is something extra that we would have to maintain, with little evidence that it is beneficial to do so. So as long as the implementation is minimally invasive, then I am open to including it. For reference, I think the differences between linux and windows is a different cmake file, a few different cpp files, and a handful of ifdefs.
And I can't guarantee that it will get the same level of effort from me in keeping it working/building, I will try but with a lower threshold for giving up than for windows/linux, but if the implementation is as isolated as windows/linux are today, then it shouldn't break often, if ever.

@Julusian Julusian changed the title Bug: CasparCG server is not compiling Bug: 2.4.x not compiling with ffmpeg 7 Oct 7, 2024
@dimitry-ishenko
Copy link
Contributor

The binary version from Ubuntu crashes on Mint - why? They are both based
on Debian.

@Behinder if you'd like, you can try Server builds from my repo: https://launchpad.net/~ppa-verse/+archive/ubuntu/casparcg

@Julusian
Copy link
Member

Julusian commented Oct 7, 2024

The binary version from Ubuntu crashes on Mint - why? They are both based
on Debian.

@Behinder if you'd like, you can try Server builds from my repo: launchpad.net/~ppa-verse/+archive/ubuntu/casparcg

or from mine https://launchpad.net/~casparcg/+archive/ubuntu/ppa

I forgot to reply to this part, its going to be some library version mismatch. The zip files are too agressive at what they include, (see #1532) but including less means requiring some packages to be manually installed on the system.
Tbh at the moment I am really tempted to stop producing those ubuntu zips, the ci for them has been broken for a bit anyway (failing on uploading for some reason)

@dimitry-ishenko
Copy link
Contributor

or from mine https://launchpad.net/~casparcg/+archive/ubuntu/ppa

@Julusian maybe some day you will accept my CEF patches, so that my builds can live in your repo 😃

@Behinder
Copy link
Author

Behinder commented Oct 8, 2024 via email

@Behinder
Copy link
Author

Linux Mint Machine

Distributor ID: Linuxmint
Description: Linux Mint 22
Release: 22
Codename: wilma

Build from https://launchpad.net/~ppa-verse/+archive/ubuntu/casparcg

[2024-10-10 14:02:20.053790] [0x000074a8cc372700] [error] 0# 0x000056B44EC94B04
1# 0x0000705756845320
2# 0x000070575689EB1C
3# 0x000070575684526E
4# 0x0000705756845320
5# 0x000056B44ED0D02C
6# 0x000056B44ED088C5
7# 0x000056B44ED0DD95
8# 0x0000705765B5097F
9# 0x000070575689CA94
10# 0x0000705756929C3C

Type "q" to close application.

There is no GUI turned on on remote machine so I this is error for launching with --noconsole flag.

Build from https://launchpad.net/~casparcg/+archive/ubuntu/ppa

[2024-10-10 14:25:05.884910] [0x00007f6f90d0d700] [error] 0# 0x0000608D0CFA0B04
1# 0x000074A8D9A45320
2# 0x000074A8D9A9EB1C
3# 0x000074A8D9A4526E
4# 0x000074A8D9A45320
5# 0x0000608D0D01902C
6# 0x0000608D0D0148C5
7# 0x0000608D0D019D95
8# 0x000074A8E8D2D97F
9# 0x000074A8D9A9CA94
10# 0x000074A8D9B29C3C

So both apps coredumped

REMEMBER: add system information log

Docker container with Ubuntu on M2Pro

Build from https://launchpad.net/~ppa-verse/+archive/ubuntu/casparcg

casparcg-server-2.4
Type "q" to close application.
zsh: abort casparcg-server-2.4

[2024-10-10 12:37:00.080767] [0x0000ffff8182d040] [error] 0# 0x0000AAAAACEB449C
1# 0x0000FFFFA23C37A0
2# 0x0000FFFF918C7628
3# 0x0000FFFF9187CB3C
4# 0x0000FFFFA23C37A0
5# 0x0000AAAAACF3F1A8
6# 0x0000AAAAACF39CA8
7# 0x0000AAAAACF3FD90
8# 0x0000FFFFA05D61E4
9# 0x0000FFFF918C597C
10# 0x0000FFFF9192BA4C

Type "q" to close application.
zsh: abort casparcg-server-2.4 --noconsole

Build from https://launchpad.net/~casparcg/+archive/ubuntu/ppa

repository was added but casparcg-server-2.4 has no installation candidate,
apt update does nothing so I guess this is repository not for docker builds.

DEBIAN Machine

Build https://launchpad.net/~ppa-verse/+archive/ubuntu/casparcg

Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
gpg: nie odnaleziono poprawnych danych w formacie OpenPGP.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 892, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 688, in addkey_func
func(**kwargs)
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 386, in add_key
return apsk.add_ppa_signing_key()
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 273, in add_ppa_signing_key
cleanup(tmp_keyring_dir)
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 234, in cleanup
shutil.rmtree(tmp_keyring_dir)
File "/usr/lib/python3.9/shutil.py", line 718, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/usr/lib/python3.9/shutil.py", line 675, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
File "/usr/lib/python3.9/shutil.py", line 673, in _rmtree_safe_fd
os.unlink(entry.name, dir_fd=topfd)
FileNotFoundError: [Errno 2] Nie ma takiego pliku ani katalogu: 'S.gpg-agent.extra'

Build from https://launchpad.net/~casparcg/+archive/ubuntu/ppa

An 'unofficial' PPA for CasparCG server (made by one of the maintainers)
Built from https://github.com/julusian/casparcg-ppa
More info: https://launchpad.net/~casparcg/+archive/ubuntu/ppa
Press [ENTER] to continue or ctrl-c to cancel adding it

gpg: keybox ,,/tmp/tmpdtf1tzk4/pubring.gpg'' utworzony
gpg: /tmp/tmpdtf1tzk4/trustdb.gpg: baza zaufania utworzona
gpg: klucz 8A520FDA9153CB31: klucz publiczny ,,Launchpad PPA for CasparCG'' wczytano do zbioru
gpg: Ogółem przetworzonych kluczy: 1
gpg: dołączono do zbioru: 1
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
gpg: nie odnaleziono poprawnych danych w formacie OpenPGP.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 892, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 688, in addkey_func
func(**kwargs)
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 386, in add_key
return apsk.add_ppa_signing_key()
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 273, in add_ppa_signing_key
cleanup(tmp_keyring_dir)
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 234, in cleanup
shutil.rmtree(tmp_keyring_dir)
File "/usr/lib/python3.9/shutil.py", line 718, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/usr/lib/python3.9/shutil.py", line 675, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
File "/usr/lib/python3.9/shutil.py", line 673, in _rmtree_safe_fd
os.unlink(entry.name, dir_fd=topfd)
FileNotFoundError: [Errno 2] Nie ma takiego pliku ani katalogu: 'S.gpg-agent.extra'

so unable to test builds on debian.
specs.zip

@Julusian
Copy link
Member

Build from launchpad.net/~casparcg/+archive/ubuntu/ppa
repository was added but casparcg-server-2.4 has no installation candidate,
apt update does nothing so I guess this is repository not for docker builds.

Its not a case of docker, but I haven't made any arm64 builds, which is what your m2 mac will be looking for.
I suspect it wouldn't even run though, because macos doesn't support the version of opengl we need. But perhaps there is some docker tooling that will run some metal translation layer that could be used. Either way, it logs when it fails to initialise opengl, and that happens much later in startup, after a bunch of logging has already happened.

So both apps coredumped

This is 'normal' behaviour when it tries and fails to find a config file, could that be what you are hitting?

so unable to test builds on debian.

tbh I didn't know that debian had any support for ppas, those are a very ubuntu convention. Its complaining something about gpg keys, no clue what to do with that tbh.

@dimitry-ishenko
Copy link
Contributor

dimitry-ishenko commented Oct 11, 2024 •
edited
Loading

Build https://launchpad.net/~ppa-verse/+archive/ubuntu/casparcg

Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
gpg: nie odnaleziono poprawnych danych w formacie OpenPGP.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 892, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 688, in addkey_func
func(**kwargs)
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 386, in add_key
return apsk.add_ppa_signing_key()
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 273, in add_ppa_signing_key
cleanup(tmp_keyring_dir)
File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 234, in cleanup
shutil.rmtree(tmp_keyring_dir)
File "/usr/lib/python3.9/shutil.py", line 718, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/usr/lib/python3.9/shutil.py", line 675, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
File "/usr/lib/python3.9/shutil.py", line 673, in _rmtree_safe_fd
os.unlink(entry.name, dir_fd=topfd)
FileNotFoundError: [Errno 2] Nie ma takiego pliku ani katalogu: 'S.gpg-agent.extra'

@Behinder try this:

sudo apt install gpg-agent
sudo mkdir -p /root/.gnupg
sudo add-apt-repository ppa:ppa-verse/casparcg
sudo apt install casparcg-server

@Behinder
Copy link
Author

Behinder commented Oct 11, 2024 via email

@Behinder
Copy link
Author

Behinder commented Oct 11, 2024 via email

@Julusian
Copy link
Member

An example config file is included in my ppa packages, at /usr/share/casparcg-server-2.4/casparcg.config.example, but casparcg won't find it by itself. I think it is looking for a casparcg.config in the current working directory, or you can specify a path to a config file.

@Behinder
Copy link
Author

Behinder commented Oct 11, 2024 via email

@dimitry-ishenko
Copy link
Contributor

dimitry-ishenko commented Oct 11, 2024 •
edited
Loading

i am launching it remotely or with --noconsole mode

What's the --noconsole option?... Where did it come from?
How are you launching the server remotely? Via ssh? If so, make sure you export the DISPLAY variable.

@Behinder
Copy link
Author

Behinder commented Oct 11, 2024 via email

@dimitry-ishenko
Copy link
Contributor

An example config file is included in my ppa packages, at /usr/share/casparcg-server-2.4/casparcg.config.example, but casparcg won't find it by itself.

FWIW, I ship .desktop file with my .deb package, which copies casparcg.config if it doesn't exist to make user experience less painful 😄

demo.mp4

@dimitry-ishenko
Copy link
Contributor

ChatGPT suggest --noconsole option is for server in headless mode without GUI my observation suggest it is true. Surprising you ask me about that now and not right after I posted :)

LOL at ChatGPT... Sorry, I've missed the first time you've mentioned the --noconsole option. The only option Server supports is the path to the casparcg.config file. There is an outstanding issue (see #1571) that prevents the server from properly reporting command-line errors.

Yes via ssh but on remote server - first I tried ssx -X and launch server
on my side but that won't work because OpenGL. ( I was able to compile mesa
with Zink driver but cannot force system to use it)

Try sshing into the remote machine without the -X option. Then run DISPLAY=:0 casparcg-server.

demo.mp4

@Behinder
Copy link
Author

Behinder commented Oct 13, 2024 via email

@Behinder
Copy link
Author

Behinder commented Oct 13, 2024 via email

@Behinder
Copy link
Author

Behinder commented Oct 13, 2024 via email

@Julusian
Copy link
Member

Julusian commented Oct 13, 2024 •
edited
Loading

chatgpt is trained on Stack Overflow so I don't uderstand your LOL here
ChatGPT did not take that info from nowhere.

If I search stackoverflow for casparcg, it comes back with 13 results. So anything chatgpt thinks it knows about casparcg from there is unlikely to have much accuracy.

Besides every server app should have headless mode whatever you called it.

I don't know what you expect from a headless mode here tbh. Seeing as the server is a command line application, it already is pretty headless. But it does require a X server, which is unavoidable if you want to be able to use html templates. And when I lasted looked a few years ago, getting a gpu handle without x was still patchy at best and inconsistent vendor support. This is probably better by now, but seeing as html support still needs it, I don't see the point in changing that.

Only supporting ubuntu is noted in the readme https://github.com/casparcg/server?tab=readme-ov-file#linux
And it is entirely possible to run on other distros, but it might require recompiling. I run it on fedora, and I know there is an arch package https://aur.archlinux.org/packages/casparcg-server

@Behinder
Copy link
Author

Behinder commented Oct 15, 2024 •
edited
Loading

I gave up but could not sleep so I spent few hours fighting with Ubuntu and digging through various wrong answer on Internet and I finally did it. So this is summary:

  1. Ocular fork of Ubuntu is painfully slow on virtual machine difficult even to install so I threw it out.
  2. I installed 22.04 server ubuntu ARM64 using qemu in UTM.
  3. I don't know if UTM did some magic or it is me who once compiled natively Gallium Zink drivers Mesa and virglrenderer, but glxinfo shows me that I have OpenGL 4.5 in Ubuntu!
  4. lxde sucks and lightdm constantly creating problems searching for XSessions called ubuntu that was not there. I used Mate2 GUI and modified .dmrc to set default session and it worked.
  5. Attached files.
    casparcg_server_aarch64-arm64.zip

Additional note: I used ffmpeg6 and option for use system ffmpeg, not sure how with 7 Ubuntu was not shipped with that version I think. Also I had to turn off CEF library completely, for some reasons it is not compatible with ARM64 architecture and that creates problem during linking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Behinder @Julusian @dimitry-ishenko