The following SQL code demonstrates the power – it returns a list of mac addresses (with some descriptive info) from the clientreports that we haven’t already recorded in the macaddr table.
WITH hostmac AS (SELECT hostname,
jsonb_array_elements(data->'network'->'nics')->'name',
jsonb_array_elements(data->'network'->'nics')->>'macaddr' AS macaddr,
data->'dmi'->'system'->>'system-serial-number' as sno
FROM clientreportjson
)
SELECT hostmac.hostname,macaddr,sno,i.item_id,i.description FROM item i
JOIN hostmac ON (
hostmac.sno = i.serial
)
WHERE macaddr::MACADDR NOT IN (SELECT macaddr from macaddr);
Experimentation with different JSON representations revealed that where you have data on multiple items of the same type (for example several NICs on the same machine), it is better to group these together in a list rather than in a hash. This makes for easier SQL processing of the data (eg using the jsonb_array_elements() operator).
]]>After reading numerous web articles and blogs on REST API design, and being confused by the differing views on what constitutes good API design, I have produced a draft API design. The API will be available in two forms – one which is completely un-authenticated but allowing only queries, the other being GSSAPI authenticated for both queries and updates. (The original intention had been for the GSSAPI authenticated tree to provide a login method issuing a short-lived token to be used by the client for subsequent operations (which wouldn’t be GSSAPI authenticated). However it turns out that GSSAPI for every operation isn’t that expensive and has the benefit of significantly simplifying the code.
The intention is to use perl-Catalyst-Action-REST to implement the API.
]]>The aim is to run the new inventory system (Tartarus) in parallel with the existing system and incrementally add features over the coming year. Order files are now sync’d across (currently nightly) to Tartarus and updating appropriately. The new system has been ported to SL7 and upgraded to PostgreSQL 9.5 (on host ‘bandama’). Thanks to Toby for building perl-DBIX-Class, and deps, to SL7.
Information on the kit purchased off the Select PC/Laptop scheme is available, via periodic reports, from the suppliers. This includes information on purchase order, serial number, primary mac address. The intention is to master mac address information in Tartarus, and not LCFG profiles – feeding these supplier reports into Tartarus will assist in automating this process.
A generic kerberised curl script (krbcurl) has been written which can be run as part of a mail server alias rule – this submits the incoming email to a CGI script (running on the Tartarus server) over https using GSSAPI authentication. The CGI script feeds the incoming mail report, based on the From address, to a manufacturer specific module which decodes (ie takes apart the MIME structure) the report and updates the Tartarus supplierreport table appropriately. This is now running live, with scheduled weekly reports from HP and manually invoked reports from Dell.
A simple centralised logging library has been written so that all code running on the Tartarus server (including CGIs) logs into one SQL event log – much like syslog (log facility, log level, message, time) . This has proven invaluable in debugging problems and should assist in debugging problems when in service.
Chris has written some more clientreport modules – reporting on firmware, PSUs, BMC info.
The upgrade to PostgreSQL 9.5 introduces the JSONB data type. Storing the clientreport JSON in a JSONB column means that we can combine queries on data in SQL tables with data held within the JSON data structure.
An important element of the new system will be the RESTful API. Currently, the invquery and invedit commands use SQL directly to perform queries and updates. This has integrity “issues”, means that changing behaviour can be laggy (code needs updated on clients), and means that other code that wants to do queries or updates must duplicate the code. A RESTful API will overcome these problems. The next significant task is to define an API and circulate for comment.
]]>Orders file processing
Order sync
Clientreport
Feeds from School database
clientreport script uses a rather crude method to determine the monitor connected to a DICE desktop. It greps through /var/log/Xorg.0.log to pick out the monitor model name, serial number and year of manufacture.
Whilst this has worked for many, many years, it is obviously rather risky. That log file is not designed to be processed and the format could change at any time. Moreover, the current code only reports on one monitor and doesn’t work for monitors attached via an nVidia graphics card.
The obvious solution was to detect monitors using the “proper” mechanism – that is, to read and decode the EDID data for each monitor. A monitor’s EDID data is usually available via /sys/class/drm/{port}/edid. On machines with nVidia cards (at least using the nVidia drivers), the EDID data is not available via /sys/class and must instead be obtained using the VBE interface. This is done using the monitor-get-edid script which is part of the monitor-edid package. However, only one monitor can be detected using this method.
Having obtained the EDID data, it can be parsed either by edid-decode, part of the xorg-x11-utils package, or monitor-parse-edid which is part of the monitor-edid package. Unfortunately, neither of these scripts display the serial number of the monitor, so it was decided to ship a patched version of the monitor-parse-edid script with clientreport which does display the serial number and other useful information.
The new clientreport script first attempts to locate the EDID data using /sys/class. If that fails it falls back to using the VBE method.
]]>
clientreport script of the current inventory system only reports on one MAC address, and even then simply derives this from the dhclient.mac resource.
As we intend, in the new inventory system, for the clientreport script to be the principal route for determining the MAC address of a machine, it must somehow obtain the MAC address directly from the Linux kernel.
There are numerous WWW postings documenting a variety of different ways to enumerate the physical network interfaces of a machine. The quickest route would appear to be to make use of the fact that only physical network interfaces have a file /sys/class/net/{iface}/device. Each of these interfaces will have a file /sys/class/net/{iface}/address the contents of which are the MAC address of that interface.
For most machines, that’s the end of the story – but not so on machines with bonded interfaces. When two interfaces are bonded, by necessity one of the pair takes on the MAC address of the other interface – so you end up with two interfaces with the same MAC address. As far as I could see, there is no way to determine the permanent MAC address of the interface whose MAC address has been altered – at least not by reading files in /sys nor by using the majority of network tools. Even the ioctl() call SIOCGIFADDR returns the modified MAC address. Once I discovered the last fact, I almost gave up with finding a solution, assuming that all tools would use either this ioctl() call, or read /sys, to determine the MAC address.
However, persistence paid off. I wondered how OCSinventory solves this problem and took at look at the source code for the OCSinventory agent. A quick peruse of this code showed that it makes use of ethtool to determine things like connection speed etc. A peek at the ethtool man page suggested ethtool --showpermaddr might do what we wanted. Fully expecting this to return the wrong address, I was pleased to find that it does indeed return the correct permanent address of an interface even if that interface is a slave in a bonded pair. So, how does it do that if ioctl(SIOCGIFADDR) and /sys return the wrong answer? It turns out that ethtool has its own private ioctl() call called SIOCETHTOOL.
]]>
In the current inventory system, clients submit their report directly to the database using an SQL UPDATE, secured by a ‘secret’ shared password. This is not exactly considered “best practice” and it was decided that, for the new Inventory system, we should update the ‘clientreport’ script to use 21st century technologies!
The proposed system is to serialise the report data using JSON and submit the JSON using an HTTPS POST to a CGI script running on the Inventory system. The POST will be made over a kerberos authenticated connection, using the client’s host principal. The CGI script can then deserialise the JSON and update the database as necessary.
You might expect such an architecture to be in common use in DICE, but surprisingly this turns out not to be the case. Certainly I failed to find any example of a Perl implementation. Fortunately Stephen had some proof-of-code snippets to start from, and a prototype was (reasonably) soon produced.
On the client side this uses Authen::Krb5 to effectively ‘kinit’ using the client’s host principal. LWP::UserAgent is then used to make the HTTPS POST, with LWP::Authen::Negotiate providing the GSSAPI connection.
On the server side is a standard lcfg-apacheconf managed web server, using mod_auth_kerb for authentication. (Obviously, a service keytab needs to be created using the lcfg-kerberos component – and this keytab needs to be owned by the apache user).
A couple of problems were encountered. Firstly, the documentation for LWP::Authen::Negotiate fails to mention that it expects the kerberos service name to be ‘HTTP@hostname’. On consideration, it’s obvious that it must make such an assumption, but it would have been nice for the documentation to state it. Secondly the documentation for mod_auth_kerb states incorrect default values for some resources. The following resources were found to work.
AuthType Kerberos KrbServiceName HTTP Krb5KeyTab /etc/httpd.keytab KrbMethodK5Passwd On KrbVerifyKDC On require valid-user]]>
So.. for F12 we have to consider how we’re going to control how services and LCFG components start at boot time. There appear to be three choices :-
After some deliberation and discussion we decided that, given that Fedora is still in a state of transition to Upstart, we should chose the least cost option (3) for F12.
]]>