Throwing Random Arguments at System Binaries: Real Segfaults vs Ticket Noise – Fedora Community Blog

Why would anyone do that?

Two and a half reasons:

1. Smoke test – You want to know if your system commands actually work, not just when you run them as the docs say, but when users (or their scripts) feed their garbage.
2. AI is excellent at generating potential edge cases, and detection systems already are all too eager to collect new tickets. I’m careful not to dump every AI found Bugzilla; I don’t want to mess up the backlog and mainly waste developer time on theoretical errors. Or should I?

Plus, segfaults don’t lie – either the system crashed or it didn’tand these are the issues that actually deserve the ticket.

Throw random arguments at system binaries until they crash

Script to do the job:

A fairly simple bash scriptvibrating with AI-generated chaos.

1. Grab all binaries of /usr/bin and /usr/sbin
2. Analyze --help for flags (--whatever, -xyou know the drill)
3. Choose random combinations of those flags (1-4 per run)
4. Feed them garbage: broken JSON/XML, binary garbage, path traversal attempts, format strings, absurdly long lines

Log only current crashes – SIGSEGV, SIGABRT, SIGILL, SIGBUS. Exit code 1 from bad args is ignored.

Core logic looks like this:

# Extract flags from --help                                            
  flags=$(timeout 3s "$bin" --help 2>&1 |                   
      grep -aoE -e '--(a-zA-Z0-9_-)+' -e '-(a-zA-Z)' |                                                    
      grep -avE 'help|version|usage')                                                                     
                                                                                                          
  # Pick random flags (1-4 of them)                                                                       
  chosen=$(echo "$flags" | shuf -n $((1 + RANDOM % 4)))                                                   
                                                                                                          
  # Add a random test file                                                                                
  fuzz_file="$WORKSPACE/$(random_pick: bad.json, random.bin, longline.txt, ...)"                          
                                                                                                          
  # Run it                                                  
  timeout 5s "$bin" $chosen $fuzz_file                                                                    
  

Script skips the obvious no go zones – package managers, rm, networking tools, editors. I am glad to see that the script is finished with the machine still answering.

Try it yourself!

Source: run-them-all

Run via test farm (how-to):

testing-farm-public request --test-type fmf \                                                
    --git-url https://forge.fedoraproject.org/quality/fmf-tests.git \                                     
    --git-ref main \
    --compose Fedora-Cloud-Base-AmazonEC2.x86_64-44-1.3 \                                                 
    --arch x86_64 \                                                                                       
    --test run-them-all \                                                                                 
    --context "force=yes" \                                                                               
    --plan /plans/all                                                                                     
  

Run locally: Just use the try-all.sh binary from the test, no requirement needed, run:

curl -sSLO https://forge.fedoraproject.org/quality/fmf-tests/raw/branch/main/system-in-use/run-them-all/try-all-binaries-help-options.sh && bash try-all-binaries-help-options.sh

My results: Test place artifacts (Fedora 44 RC compose 1.3, ~950 binaries tested)

What I found

grub2-mkrescue (bootloader), pearl (half the system depends on it), cf of groff (man pages break without it).

What coredumpctl caught:

TIME                           PID UID GID SIG     COREFILE EXE                      SIZE
  Mon 2026-04-20 12:08:09 UTC  52378   0   0 SIGABRT present  /usr/bin/edgepaint      80.7K               
  Mon 2026-04-20 12:08:22 UTC  58926   0   0 SIGSEGV present  /usr/bin/eqn            57.8K
  Mon 2026-04-20 12:08:41 UTC  77687   0   0 SIGSEGV present  /usr/bin/gdbm_dump      23.9K               
  Mon 2026-04-20 12:09:01 UTC  97901   0   0 SIGSEGV present  /usr/bin/grub2-mkrescue 64.7K
  Mon 2026-04-20 12:09:08 UTC 106904   0   0 SIGSEGV present  /usr/bin/gtshapprox     48.3K               
  Mon 2026-04-20 12:10:15 UTC 161697   0   0 SIGABRT present  /usr/bin/perl5.42.1       94K               
  

Outline:

• border paint (graphviz) – 4× SIGABRT
• gtsapprox (graphviz) – 3× SIGSEGV
• pearl – 4× SIGABRT
• cf (coarse) – 1× SIGSEGV
• gdbm_dump (gdbm) – 1× SIGSEGV
• grub2-mkrescue (grub2) – 1× SIGSEGV

Players

Run it on your side to see the players in action. I’m curious to hear your thoughts on these great findings.

MY CRASHES:
COMMAND: /usr/bin/edgepaint -s -o --random_seed --angle --lightness -v --accuracy --share_endpoint 
COMMAND: /usr/bin/edgepaint -v -s --accuracy --color_scheme -o  /root/fuzz_lab/empty.dat
COMMAND: /usr/bin/edgepaint --share_endpoint -o --angle -v -s --random_seed --accuracy --color_scheme  -
COMMAND: /usr/bin/edgepaint -s --lightness --share_endpoint -o -v --angle --random_seed 
COMMAND: /usr/bin/efibootdump --guid -g -f  /root/fuzz_lab/large.dat
COMMAND: /usr/bin/eqn -C -f -M -d -v -m -T -s  /root/fuzz_lab/fake.png
COMMAND: /usr/bin/gdbm_dump --format  -
COMMAND: /usr/bin/grub2-file --is-x86-knetbsd  /root/fuzz_lab/longline.txt
COMMAND: /usr/bin/grub2-file --is-x86-knetbsd  /root/fuzz_lab/bad.json
COMMAND: /usr/bin/gtshapprox --flat -c  /root/fuzz_lab/fake.jpg
COMMAND: /usr/bin/gtshapprox -v -f --verbose -c  /root/fuzz_lab/cmd.txt
COMMAND: /usr/bin/gtshapprox -n -h --flat -l --closed -v  /root/fuzz_lab/gzip.dat
COMMAND: /usr/bin/gtshapprox --flat --number --log --cost --closed  -
COMMAND: /usr/bin/gtshapprox --log -l -f --keep  -
COMMAND: /usr/bin/mkfs.xfs -q  /root/fuzz_lab/bad_utf8.txt
COMMAND: /usr/bin/mkfs.xfs -K -L -m  /root/fuzz_lab/paths.txt
COMMAND: /usr/bin/perl -V -F -T -S -u -p -f -E -W -I  /root/fuzz_lab/bad.json
COMMAND: /usr/bin/perl -F -e -n -a -l -u -c  -
COMMAND: /usr/bin/perl -S -u -s -E -I -l 
COMMAND: /usr/bin/perl -s -p -n -u -D -E -d 
COMMAND: /usr/bin/tree -P --filelimit --matchdirs --hyperlink -i  /root/fuzz_lab/paths.txt

Look, these are edge cases. No one actually runs edgepaint --wtf malformed.json in prod. But segfaults are segfaults – the binary should dance with “invalid option” or “bad input”, not dump core.

now what?

So I have a lot of crashes. Some in critical components. All reproducible.

File errors for all of them? That’s a lot of BZ tickets for “yeah hm it crashes if you feed it random garbage with weird flags”. Developers have better things to do.

Ignore them? They are real bugs. And some of these are in grub2 and perl – not exactly throwaway packages.

Still figuring it out.

My Dilemma During Test on Fedora 44 RC compile 1.X