Welcome to ivulncheck’s documentation!

Contents:

Indices and tables

Ivulncheck

What is ivulncheck?

Ivulncheck is a security tool to centralize the search of vulnerabilities and fixes concerning packages on a cluster.

A procedure is needed to initialize the API (restore an initial state of the MongoDB database, download ubuntu-cve-tracker tool, deploy CVE-Search modified API)

First, an agent script must be deployed to detect installed packages on the cluster (only Ubuntu and Debian are supported). This script generated a json formatted file and send it to ther API. Then, several commands must be executed to search vulnerabilities and determine available security updates (called here fixes). Four principals sources are used :

  • debsecan for Debian hosts,
  • ubuntu-cve-tracker for Ubuntu hosts,
  • CVE-Search for custom installed application (manually configuration needed),
  • madison-lite (to complete missing fixes due to possibly outdated sources, missing sources or unreferenced vulnerabilities)

Eventually, you can get informations (installed packages, monitored hosts, known vulnerabilities an available fixes) directly from the API Or a Web Interface provide a graphical rendering of these informations.

Installation

You have to set a compilation environment for debian unstable with sbuild, fakeroot

Commit last changes
git commit -a
Tag the new version to be compile
git tag <version>
Get in the debian branch
git checkout debian/unstable
Merge last modifications from tagged branch
git merge -X theirs <version>
Edit debian/changelog file with the new version and commit your changes in debian/unstable branch
git commit -a -m “Changing version to <version>”
Generate original archive
./debian/rules gen-orig-xz
Compile the package
gbp buildpackage

The .deb package goes to ../build-area/ivulncheck-xxx_<version>_all.deb

  • Installation of the package dpkg -i ivulncheck-api_x.x.x-1_all.deb dpkg -i ivulncheck-web_x.x.x-1_all.deb apt-get -f update
You have to start mongodb server to be able to use ivulncheck
service mongodb start
Or
systemctl start mongodb

If you do not need mongodb authentication, you can delete username and password lines from /etc/ivulncheck/ivulncheck-api.ini

If you want it, you can configure mongodb authentication and change the default password in /etc/ivulncheck/ivulncheck-api.ini
mongo > use admin > db.createUser({ user: “admin”, pwd: “<password>”, roles: [{ role: “userAdminAnyDatabase”, db: “admin”}]}) > db.createUser({user: “ivulncheck”, pwd: “ivulncheck”, roles: [ { role: “readWrite”, db: “hostdb” }, { role: “readWrite”, db: “vulndb” }, { role: “readWrite”, db: “fixdb” } ] }) > exit
You have to enable auth in /etc/mongodb.conf
sudo sed -i ‘s/^#auth = true/auth = true/’ /etc/mongodb.conf
You need to restart mongodb
service mongodb restart
Or
systemctl restart mongodb

To log back as admin you can do $ mongo $ > use admin $ > db.auth(‘admin’, “<password>”)

You can change ivulncheck password with
> db.changeUserPassword(“ivulncheck”, “<new_password>”)

API needs apache2 to work and ivulncheck-api conf is added to an existing virtualhost and an alias is created. So you need apache server to be installed and running

service apache2 start
  • MongoDB

You need to restore an initial database state before being able to use ivulncheck You can launch ivulncheck_mongodb script in /usr/share/ivulncheck-api directory (edit the password if needed) Or you can restore database manually

mongorestore -u ivulncheck -p <password> –authenticationDatabase=admin –gzip /usr/share/ivulncheck-api/emptydb

Be careful, some old mongodb version does not support –gzip option so you may have to deflate all files in emptydb directory.

  • Apache2
You need to create users for connectin to API and Web interface.
sudo htpasswd /etc/ivulncheck/htpasswd_agent ivulncheck_agent sudo htpasswd /etc/ivulncheck/htpasswd_api ivulncheck_api sudo htpasswd /etc/ivulncheck/htpasswd_web ivulncheck_web

And do not forget to update configuration files in /etc/ivulncheck with new credentials.

  • SSL

You can add SSL certificate to enable HTTPS Certificate’s files path are

/etc/apache2/ssl/server.pem /etc/apache2/ssl/server.key /etc/apache2/ssl/ca.pem
  • Ubuntu CVE Tracker

Only if you want to manage Ubuntu hosts

ubuntu-cve-tracker tool need to be initialized.

(Not yet fully functionnal) You can launch ivulncheck_download script in /usr/share/ivulncheck-api directory

Or you can manually set it
bzr branch lp:~ubuntu-security/ubuntu-cve-tracker/master /var/lib/ivulncheck-api/ubuntu-cve-tracker
Edit ubuntu-cve-tracker/scripts/packages-mirror and changed those 3 paths
outPath=/var/lib/ivulncheck-api/ubuntu-cve-tracker/ubuntu-archive-metadata … debianPath=/var/lib/ivulncheck-api/ubuntu-cve-tracker/debian-testing-archive-metadata … partnerPath=/var/lib/ivulncheck-api/ubuntu-cve-tracker/partner-archive-metadata
And launch those 2 commands
cd /var/lib/ivulncheck-api/ubuntu-cve-tracker ./scripts/packages-mirror ./scripts/packages-mirror -p
And do not forget to restore the right rights…
chown -R ivulncheck:ivulncheck /var/lib/ivulncheck-api/ubuntu-cve-tracker/
  • Madison-lite

To execute madison-lite tool offline (a lot faster than rmadison) you need to download part of Debian and Ubuntu repositories information An API function exists to do this

Launch this URL http://<ivulncheck-api-host>/ivulncheck_api/cmd/fake_repos

Or this command with CLI interface
ivulndownload -fakerepos

This will download only information about installed hosts suite and release

  • CVE Search

CVE Search tool can be installed to make CVE link offline and faster.

This tool is also useful for vulnerability checks based on CPE (Common Product Enumeration) if you need to monitor products which are not Debian or Ubuntu package.

This functionality required a patched version of CVE-Search which I made and I have not yet published it or fork or made a feature request (because it was the first approach to check vuln but it is not convenient. There are a lot of vulnerabilities missing or false positives)

I will add this feature in a next release

Utilization

An agent is deployed on all monitored servers and works every day. It detects the OS version , lists installed packages and sends those information in JSON format to the server. (delayed on 1 hour to not overload the server)

The API full a database with those packages and checks vulnerabilities and available fixes on them. * DebSeCan approach :

  • debsecan_vulns: checks vulnerabilities on Debian packages and tags fixed packages from debsecan tools
  • debsecan_fixes: determines fixes from tags and on comparing installed version with fixed version
  • Ubuntu approach (same as DebSecAn but with Ubuntu-CVE-Tracket tool) :
  • ubuntu_vulns: checks vulnerabilities on Ubuntu packages and tags fixed packages from ubuntu-cve-tracker tools
  • ubuntu_fixes: determines fixes from tags and on comparing installed version with fixed version
*Madison Lite approach :
  • madison_fixes: checks residual fixes on querying official security repositories from Debian and Ubuntu (downloaded locally for faster search)

residual fixes that are not detected by neither DebSecAn nor Ubuntu-CVE-Tracker can be minor fixes or fixes that are not concerned by a CVE

The MongoDB structure is:

1 Database hostdb with collections hosts and packages where are the list of installed packages from hosts

1 Database vulndb with collections pkg2cpe (an index of correspondance between CPE and package) and vulns where are all vulnerabilities with tags. The key index are <cve_id> and <package_name>

1 Database fixdb with collection fixes where are the fixes found from vulnerabilities

  • Index page

This page shows global stats on vulnerabilities and fixes.

  • Hosts page

This page shows hosts in database and links to the corresponding host page.

  • Vulns page

Cette page présente toutes les vulnérabilités avec des statistiques et des filtres de recherches. * Fixes page

Cette page présente tous les patchs avec des statistiques et des filtres de recherches. * Host page

Cette page présente les statistiques d’un hôte et les liens vers ces vulnérabilités et patchs.

Commands send request to the API and each command has his usage

ivulnlist: lists MongoDB info with only skip and limit options

ivulninsert: inserts manually an host with a JSON formated package list

ivulncheck: launches checks independently (3 approaches)

ivulndownload: download repositories from Debian and Ubuntu websites (only for detected installed versions)

ivulnpurge: purge MongoDB collections (but keep indexes)

API functions return JSON responses

  • Search functions

http://<api_server>/ivulncheck_api/hosts

Méthodes supportées GET et POST

http://<api_server>/ivulncheck_api/pkgs

Méthodes supportées GET et POST

http://<api_server>/ivulncheck_api/vulns

Méthodes supportées GET et POST

http://<api_server>/ivulncheck_api/fixes

Méthodes supportées GET et POST

http://<api_server>/ivulncheck_api/host/<hostname>

Supported method GET

http://<api_server>/ivulncheck_api/host/pkgs/<hostname>

Supported method GET

http://<api_server>/ivulncheck_api/vulns/<hostname>

Supported method GET

http://<api_server>/ivulncheck_api/host/fixes/<hostname>

Supported method GET

http://<api_server>/ivulncheck_api/host/stats/<hostname>

Supported method GET

http://<api_server>/ivulncheck_api/pkg/<pkgvers>

Méthodes supportées GET

http://<api_server>/ivulncheck_api/vuln/<vuln_id>

Supported method GET

http://<api_server>/ivulncheck_api/os_versions

Supported method GET

http://<api_server>/ivulncheck_api/hosts/stats

Supported method GET

http://<api_server>/ivulncheck_api/vulns/stats

Supported method GET

http://<api_server>/ivulncheck_api/fixes/stats

Supported method GET

http://<api_server>/ivulncheck_api/vulns/<hostname>

Supported method GET

http://<api_server>/ivulncheck_api/host/fixes/<hostname>

Supported method GET

http://<api_server>/ivulncheck_api/host/stats/<hostname>

Supported method GET

http://<api_server>/ivulncheck_api/pkg/<pkgvers>

Méthodes supportées GET

http://<api_server>/ivulncheck_api/vuln/<vuln_id>

Supported method GET

http://<api_server>/ivulncheck_api/os_versions

Supported method GET

http://<api_server>/ivulncheck_api/hosts/stats

Supported method GET

http://<api_server>/ivulncheck_api/vulns/stats

Supported method GET

http://<api_server>/ivulncheck_api/fixes/stats

Supported method GET

  • Command functions

http://<api_server>/ivulncheck_api/cmd/check/<collection>

Supported method GET

http://<api_server>/ivulncheck_api/cmd/purge/<action>

Supported method GET

Insert function

http://<api_server>/ivulncheck_api/insert/host

Supported method PUT