Add bluez test(s)

[terriko]

#232 added a checker for bluez that currently has no tests. It could use a "fake" file test as well as a test against a known bluez package.

You can read more about adding tests in the test directory README.md here: https://github.com/intel/cve-bin-tool/blob/master/test/README.md

[terriko]

There's two types of tests that should be added. Here's instructions for each:

1. Adding new CVE Mapping tests
2. Adding signature tests against real files

Both of these are beginner-suitable tasks. This is a new checker and not very well-tested, so it's possible you will discover an issue with the checker itself during testing. If you do, please mention it here!

[k-udupa2000]

I tried adding a file test to bluez package(version 5.50)
Code:
@unittest.skipUnless(os.getenv("LONG_TESTS") == "1", "Skipping long tests")
def test_bluez_5_50_3(self):
""" Test detection on bluez 5.50.3 fedora bluez """
self._file_test(
"http://mirrors.kernel.org/fedora/releases/29/Workstation/x86_64/os/Packages/b/",
"bluez-5.50-3.fc29.x86_64.rpm",
"bluetoothctl",
"5.50",
)
But I am facing the following error.

1. The dictionary doesnt contain 5.50 version.
   AssertionError: '5.50' not found in {'': defaultdict(<class 'list'>, {'CVE-2016-7837': CVE(number='CVE-2016-7837',
   version='5.41', severity='HIGH'), 'CVE-2016-9797': CVE(number='CVE-2016-9797', version='5.42', severity='MEDIUM'),
   'CVE-2016-9798': CVE(number='CVE-2016-9798', version='5.42'....

I checked cvedetails website and found that 5.50 is listed under CVE-2018-10910.

Is there a way to update the dictionary?
https://www.cvedetails.com/cve/CVE-2018-10910/

Could someone guide me through this.

[terriko]

Okay, let's debug. You want to see if 5.50 exists, so let's make a little file that checks only that (and skips the binary scanning).

vendor,package,version
bluez,bluez,5.50

If you run that against csv2cve, what do you get? The command looks like this

python -m cve_bin_tool.csv2cve blueztest.csv

(where blueztest.csv is whatever you called that little file)

You can also look directly into the database with sqlite3 and see what the records look like to see if there's anything under 5.50. sqlite3 ~/.cache/cve-bin-tool/nvd.vulnerabilities.db

then call select * from nvd_data where cve_number="CVE-2018-10910"; to see what it finds. (It's... not good)

Spoiler: when I run equivalent tests on my dev branch #285 I get a different result than if I run it on master, so there's going to be an issue there.

BUT! That error message isn't telling you that 5.50 doesn't exist in the database, it's telling you that the binary it scanned didn't appear to be bluetoothctl 5.50. So you might want to also try running the command line against it to see what you're getting:

python -m cve_bin_tool.cli -x ~/tmp/bluez-5.50-3.fc29.x86_64.rpm

Take a look at the line "Known CVES in ..." and I think you'll see why it's not finding the right bugs!

[terriko]

So next steps: why is it not finding 5.50? Take a look at checkers/bluez.py and see what it's doing, then maybe run strings against that rpm to see if you can find the version string yourself (strings ~/tmp/bluez-5.50-3.fc29.x86_64.rpm | grep 5.50 might be a good place to start.)

[k-udupa2000]

Thanks for the explanation!
I ran the command

strings ~/tmp/bluez-5.50-3.fc29.x86_64.rpm | grep versionNo
for versionNo = 5.42, 5.46 and found that it returned some matches.

For the following code,
self._file_test(
"http://mirrors.kernel.org/fedora/releases/29/Workstation/x86_64/os/Packages/b/",
"bluez-5.50-3.fc29.x86_64.rpm",
"bluetoothctl",
"",
)

I checked in the cvedetails website, these CVE numbers match with the versions present.

I have single doubt though, I have kept empty string for the version number. Does that mean it will return all the versions containing vulnerabilites for the given package?

[terriko]

I'd say it's a bug in the bluez checker if it's returning an empty string

[terriko]

And, bug aside, I'm not entirely sure what it does with the empty string. it depends on whether the function we're using for version comparison thinks that "" is greater than, less than, or equal to stuff in the database. You can check that on the command line if you're curious!

[terriko]

oh, and we could also make that behaviour more intentional if we want to, so that the tool throws an error or something. There's a bit of a thought about that in #263 that we haven't quite fleshed out.

[SinghHrmn]

Hi @terriko, I tried adding some CVE mapping test for bluez. But I guess there is some error can you help me ?

def test_bluez_5_42(self):
        """Scanning test-bluez-5.42.out"""
        self._binary_test(
            "test-bluez-5.42.out",
            "bluez",
            "5.42",
            [
                #for known CVE
                "CVE-2016-9797",
                "CVE-2016-9798",
                "CVE-2016-9799",
                "CVE-2016-9800",
                "CVE-2016-9801",
                "CVE-2016-9802",
                "CVE-2016-9803",
                "CVE-2016-9804",
                "CVE-2016-9917",
                "CVE-2016-9918",
            ],
            [
                #for older version
                "CVE-2016-7837",
                
            ],
        )

When I tried running this code I got the following error

rm *.out
gcc -o test-nss-3.45.out test-nss-3.45.c
gcc -o test-curl-7.59.0.out test-curl-7.59.0.c
gcc -o test-expat-2.0.1.out test-expat-2.0.1.c
gcc -o test-nss-3.35.out test-nss-3.35.c
gcc -o test-png-1.6.26.out test-png-1.6.26.c
gcc -o test-png-1.6.36.out test-png-1.6.36.c
gcc -o test-libgcrypt-1.7.6.out test-libgcrypt-1.7.6.c
gcc -o test-gnutls-cli-2.3.11.out test-gnutls-cli-2.3.11.c
gcc -o test-ffmpeg-4.1.4.out test-ffmpeg-4.1.4.c
gcc -o test-libjpeg-turbo-2.0.1.out test-libjpeg-turbo-2.0.1.c
gcc -o test-openssl-1.0.2g.out test-openssl-1.0.2g.c
gcc -o test-zlib-1.2.8.out test-zlib-1.2.8.c
gcc -o test-gnutls-serv-2.3.11.out test-gnutls-serv-2.3.11.c
gcc -o test-png-1.4.11.out test-png-1.4.11.c
gcc -o test-bluez-5.42.out test-bluez-5.42.c
gcc -o test-sqlite-3.12.2.out test-sqlite-3.12.2.c
gcc -o test-xml2-2.9.2.out test-xml2-2.9.2.c
gcc -o test-xml2-2.9.0.out test-xml2-2.9.0.c
gcc -o test-xerces-3_1_1.out test-xerces-3_1_1.c
gcc -o test-node-9.3.0.out test-node-9.3.0.c
gcc -o test-tiff-4.0.9.out test-tiff-4.0.9.c
gcc -o test-curl-7.57.0.out test-curl-7.57.0.c
gcc -o test-openssh-7.9.out test-openssh-7.9.c
gcc -o test-icu-3.8.1.out test-icu-3.8.1.c
gcc -o test-curl-7.34.0.out test-curl-7.34.0.c
gcc -o test-ffmpeg-4.1.3.out test-ffmpeg-4.1.3.c
gcc -o test-openssl-1.1.0g.out test-openssl-1.1.0g.c
gcc -o test-kerberos-5-1.15.1.out test-kerberos-5-1.15.1.c
gcc -o test-systemd-239.out test-systemd-239.c
Skip NVD database updates.
cve_bin_tool.Scanner - INFO - Checkers loaded: bluez, curl, expat, ffmpeg, gnutls, icu, kerberos, libcurl, libgcrypt, libjpeg, libnss, libtiff, node, openssh, openssl, png, sqlite, systemd, xerces, xml2, zlib
F
======================================================================
FAIL: test_bluez_5_42 (test.test_scanner.TestScanner)
Scanning test-bluez-5.42.out
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mastervulcan/gsoc-cve-bin-tool/cve-bin-tool/test/test_scanner.py", line 148, in test_bluez_5_42
    "CVE-2016-7837",
  File "/home/mastervulcan/gsoc-cve-bin-tool/cve-bin-tool/test/test_scanner.py", line 94, in _binary_test
    self.assertIn(package, list(cves.keys()))
AssertionError: 'bluez' not found in []

----------------------------------------------------------------------
Ran 1 test in 4.746s

FAILED (failures=1)

Can you tell me why it is unable to find bluez.

[SinghHrmn]

I've got the list of CVE from cached CVE data base.

sqlite> select * from nvd_data where Vendor_Name ="bluez";
CVE-2008-2374|bluez|bluez_libs|10|6.4|HIGH|3.30
CVE-2008-2374|bluez|bluez_utils|10|6.4|HIGH|3.33
CVE-2016-7837|bluez|bluez|1.8|5.9|HIGH|5.41
CVE-2016-9797|bluez|bluez|3.9|1.4|MEDIUM|5.42
CVE-2016-9798|bluez|bluez|3.9|1.4|MEDIUM|5.42
CVE-2016-9799|bluez|bluez|3.9|1.4|MEDIUM|5.42
CVE-2016-9800|bluez|bluez|3.9|1.4|MEDIUM|5.42
CVE-2016-9801|bluez|bluez|3.9|1.4|MEDIUM|5.42
CVE-2016-9802|bluez|bluez|3.9|1.4|MEDIUM|5.42
CVE-2016-9803|bluez|bluez|3.9|1.4|MEDIUM|5.42
CVE-2016-9804|bluez|bluez|3.9|1.4|MEDIUM|5.42
CVE-2016-9917|bluez|bluez|3.9|3.6|HIGH|5.42
CVE-2017-1000250|bluez|bluez|2.8|3.6|MEDIUM|5.46

[terriko]

A result of [] likely means no checkers matched test-bluez-5.42.out, so you should take a look at the checker and your file to make sure the signature matches.

There's not much in the way of signature in bluez.py, so the thing it's looking for will match this regex:

regex = [r"bluetoothctl: ([5]+\.[0-9]+\.[0-9]+)"]

At a guess, it's not finding it because bluetoothctl: 5.42 won't match that regex (it would need to be 5.42.0 or something -- there's two dots in there). Looking at it, I'd guess that the regex may be wrong and needs to be fixed, since it's clear from your cve database info that lots of those versions only have one dot in them.

[SinghHrmn]

@terriko thanks for the great explanation. I'll try updating regex and see if I can find solution to the above problem

[SinghHrmn]

@terriko I tried to fix the previous issue. Updated the regex and also the name of the binary file.

regex = [r"bluetoothctl: ([5]+\.[0-9]+)"]

According to the checker it was finding libbluetooth.so in the filename or bluetoothctl as the last filename. So adding libbluetooth.so in the filename worked perfectly.

(venvgsoc) mastervulcan@DESKTOP-DMEM457:~/gsoc-cve-bin-tool/cve-bin-tool$ python -m unittest test.test_scanner.TestScanner.test_bluez_5_42
rm *.out
gcc -o test-bluetoothctl-5.42libbluetooth.so.out test-bluetoothctl-5.42libbluetooth.so.c
gcc -o test-curl-7.59.0.out test-curl-7.59.0.c
gcc -o test-expat-2.0.1.out test-expat-2.0.1.c
gcc -o test-nss-3.35.out test-nss-3.35.c
gcc -o test-png-1.6.26.out test-png-1.6.26.c
gcc -o test-png-1.6.36.out test-png-1.6.36.c
gcc -o test-libgcrypt-1.7.6.out test-libgcrypt-1.7.6.c
gcc -o test-gnutls-cli-2.3.11.out test-gnutls-cli-2.3.11.c
gcc -o test-ffmpeg-4.1.4.out test-ffmpeg-4.1.4.c
gcc -o test-libjpeg-turbo-2.0.1.out test-libjpeg-turbo-2.0.1.c
gcc -o test-openssl-1.0.2g.out test-openssl-1.0.2g.c
gcc -o test-zlib-1.2.8.out test-zlib-1.2.8.c
gcc -o test-gnutls-serv-2.3.11.out test-gnutls-serv-2.3.11.c
gcc -o test-png-1.4.11.out test-png-1.4.11.c
gcc -o test-sqlite-3.12.2.out test-sqlite-3.12.2.c
gcc -o test-xml2-2.9.2.out test-xml2-2.9.2.c
gcc -o test-xml2-2.9.0.out test-xml2-2.9.0.c
gcc -o test-xerces-3_1_1.out test-xerces-3_1_1.c
gcc -o test-node-9.3.0.out test-node-9.3.0.c
gcc -o test-tiff-4.0.9.out test-tiff-4.0.9.c
gcc -o test-curl-7.57.0.out test-curl-7.57.0.c
gcc -o test-openssh-7.9.out test-openssh-7.9.c
gcc -o test-icu-3.8.1.out test-icu-3.8.1.c
gcc -o test-nss-3.45.out test-nss-3.45.c
gcc -o test-curl-7.34.0.out test-curl-7.34.0.c
gcc -o test-ffmpeg-4.1.3.out test-ffmpeg-4.1.3.c
gcc -o test-openssl-1.1.0g.out test-openssl-1.1.0g.c
gcc -o test-kerberos-5-1.15.1.out test-kerberos-5-1.15.1.c
gcc -o test-systemd-239.out test-systemd-239.c
Skip NVD database updates.
cve_bin_tool.Scanner - INFO - /home/mastervulcan/gsoc-cve-bin-tool/cve-bin-tool/test/binaries/test-bluetoothctl-5.42libbluetooth.so.out is bluetoothctl 5.42
cve_bin_tool.Scanner - INFO - Known CVEs in version 5.42
cve_bin_tool.Scanner - INFO - CVE-2016-9797, CVE-2016-9798, CVE-2016-9799, CVE-2016-9800, CVE-2016-9801, CVE-2016-9802, CVE-2016-9803, CVE-2016-9804, CVE-2016-9917, CVE-2017-1000250, CVE-2018-10910
.
----------------------------------------------------------------------
Ran 1 test in 5.109s

OK

But when I tried running on a deb file the checker failed to recognise the version of bluez package

(venvgsoc) mastervulcan@DESKTOP-DMEM457:~/gsoc-cve-bin-tool/cve-bin-tool$ python -m cve_bin_tool.cli -x ../testFilesCVE-bin-tool/bluez_5.48-0ubuntu3.2_amd64.deb 
cve_bin_tool.CVEDB - INFO - Using cached CVE data (<24h old). Use -u now to update immediately.
cve_bin_tool.Scanner - INFO - Checkers: bluez, curl, expat, ffmpeg, gnutls, icu, kerberos, libcurl, libgcrypt, libjpeg, libnss, libtiff, node, openssh, openssl, png, sqlite, systemd, xerces, xml2, zlib
cve_bin_tool - INFO - None
cve_bin_tool.Scanner - ERROR - error: no version info for 'bluetoothctl'
cve_bin_tool.Scanner - ERROR - error: no version info for 'bluetoothctl'
cve_bin_tool - INFO - 
cve_bin_tool - INFO - Overall CVE summary: 
cve_bin_tool - INFO - There are 0 files with known CVEs detected

I guess we might have to replace the regex with the correct one anyway.

[terriko]

In some cases, you won't replace the regex so much as add to it -- you can have as many regexes as you want to support different distros if they're different. (and experience says yes, sometimes they are.)

[terriko]

Still looking for a good test here. Feel free to work on this even if someone else is already working on it -- there is no harm in havnig more than one file test for bluez and unless people happen to choose absolutely identical files, we can merge multiple solutions here.