Filter machines by power address, serial number (BMC, iLO, DRAC, etc.)

[James Vaughn]

This comes from a customer request.

As per title, would it be possible to filter machines by power address (BMC, iLO, iDRAC, etc.)?

In case anybody else wants this feature and comes across this request, at the moment, I can think of a couple of workarounds (of varying suitability):

- Use tags. It isn't possible to partially match a tag (e.g. tag "192.168.1.10" won't be matched by a search for "192"), so it makes most sense to implement some sort of tagging scheme to achieve this, e.g. if you have a BMC network of 192.168.1.0/24 and a machine with a power address of 192.168.1.10, you could have "power:192.168.1.0/24" and "power:192.168.1.10" as tags for that machine.

- Use `jq` in the CLI. This was thrown together quickly, but the following could be used to return the power parameters and "system_id" of every machine managed by MAAS:

```
for i in $(maas "$MAAS_USER" machines read | jq -r '.[].system_id'); do maas <LOGIN> machine power-parameters "$i" | jq ". + {\"system_id\": \"$i\"}"; done | jq -s
```

Alternatively, to filter by particular attributes, such as "power_type":

```
for i in $(maas "$MAAS_USER" machines read | jq -r '.[] | select(.power_type == "moonshot") | .system_id'); do maas <LOGIN> machine power-parameters "$i" | jq ". + {\"system_id\": \"$i\"}"; done | jq -s
```

These would return:

```
[
  {
    "power_pass": "notapass",
    "power_user": "notauser",
    "power_address": "10.188.127.24",
    "power_hwaddress": "52:54:00:5e:28:8f",
    "system_id": "ybb6aq"
  },
  {
    "power_pass": "notauser2",
    "power_user": "notapass2",
    "power_address": "10.188.127.25",
    "power_hwaddress": "52:54:00:71:ef:f3",
    "system_id": "8wyxah"
  }
]
```

Which can then further be filtered with `jq`. Of course, given that a `power-parameters` query is made for every MAAS machine, this is slow. Additionally, "system_id" is not a particularly human-readable attribute; including "fqdn", for example, would be much more useful. However, this is beyond the scope here.

Reference to a case link: https://canonical.lightning.force.com/lightning/r/Case/5004K000009Vg1LQAS/view

EDIT (2021-08-17): I've tested a MAAS 2.7.2 installation (STS MAAS) and found that if filtering by power address or serial number works. It therefore seems that there might be a regression here.

[James Vaughn (jmcvaughn)]

Edited original comment to note that this used to work in 2.7.2. Removed "feature" tag, updated title to include serial number filtering.

[Amy Pattanasethanon (amylily)]

Hi James,

Can you provide us some screenshots/URL or examples of how it work in 2.7? Just so we can see the reference :)

[Adam Collard (adam-collard)]

Note - a bit hidden, but in the UI, one can search by power_*type* (e.g. IPMI, LXD, Virsh ...) by using "power_type:IPMI" in the search bar. This unfortunately doesn't extend to power address, which I know was the original request - just think this might help folks.

[Adam Collard (adam-collard)]

@James - admin users can get power_parameters for all of the machines in MAAS by using

maas "$MAAS_USER" machines power-parameters

So instead of this monstrosity

```
for i in $(maas "$MAAS_USER" machines read | jq -r '.[].system_id'); do maas $MAAS_USER machine power-parameters "$i" | jq ". + {\"system_id\": \"$i\"}"; done | jq -s
```

You can simply have `maas "$MAAS_USER" machines power-parameters | jq` to see all of them (in about 25x less time on my test)

And instead of

```
for i in $(maas "$MAAS_USER" machines read | jq -r '.[] | select(.power_type == "moonshot") | .system_id'); do maas <LOGIN> machine power-parameters "$i" | jq ". + {\"system_id\": \"$i\"}"; done | jq -s
```

to select for a power_type you can have

`maas "$MAAS_USER" machines read | jq -r '.[] | select(.power_type == "moonshot") | "id=" + .system_id'| xargs -r maas "$MAAS_USER" machines power-parameters`

which again is much more performant

[James Vaughn (jmcvaughn)]

@Adam Thanks for both your responses; really appreciate it. I don't know how I missed `maas LOGIN machines power-parameters`...

@Amy Apologies for the delay getting back to you; please see my next comment (https://bugs.launchpad.net/maas/+bug/1938927/comments/6) for the requested screenshots.

I realise that my findings aren't clearly stated here (discussed internally instead), so starting again below.

With MAAS 2.7.2 (STS MAAS, 2.7.2 (8283-g.72f2ee59d-0ubuntu1~18.04.1)) I found that filtering by power IP address worked, then filtering by serial didn't. I reloaded a few times and it eventually filtered by serial successfully. Despite connecting over our VPN, I don't believe that connectivity was the issue given that when MAAS goes down or the connection is lost, the web UI immediately displays "Connection lost, reconnecting...", but I could be wrong.

I then tried searching for the power address in MAAS 3.0 and this time it worked. Likely by pure coincidence, when I tried this to create the bug report initially. I'm using the 3.0/stable snap, (3.0.0-10029-g.986ea3e45, 2021-06-22 (15003)). Looking at my command history I last restarted MAAS (`sudo snap restart maas`) on 2021-08-12, which is right in the middle of testing/working on this case, so it is possible that a restart resolved the issue.

I couldn't test filtering by serial locally as the machine is not a real one (and thus couldn't be commissioned to obtain a serial number).

Please find attached screenshots from 2.7.2 and from 3.0.0.

From the customer's end the following still needs to be done based on the above:

- Restart MAAS and try again

- Clear browser cache/cookies/etc. and try again

- Try a different browser

[James Vaughn (jmcvaughn)]

Customer update:

- They're using 3.0.0-10029-g.986ea3e45 15003 like I am

- They've restarted the region controller to no avail

- They've tried multiple browsers (Chrome and Firefox)

Looking at my own system I see that no logging is performed when making queries. Looking at the documentation I'm not seeing any way to enable this. Tweaking nginx.conf results in changes being made on restart. Do we have any other methods to enable debug logging or equivalent?

[Huw Wilkins (huwshimi)]

Hi James, power parameters and the metadata that includes the serial are not fetched when we load the machine list but are fetched when you view a machine's details page (and a few other places in the UI).

My guess is that is why you're only sometimes able to search by those parameters.

Adam, I'm not sure what the solution is for the machine search. Do we need metadata and power parameters to be included in the machine list response? We would still need to add the mappings for power parameters and metadata to the frontend machine search, but there's not much point unless that data is available.

[James Vaughn (jmcvaughn)]

Hi Huw,

Thanks for confirming; I've tested this and found the same. I've added the feature tag bag in as a result.

Just a final note/correction r.e. tags from the bug description. From further testing and a proper look at https://maas.io/docs/snap/2.9/ui/tags-and-annotations#heading--name-tags, MAAS only supports alphanumeric characters, underscores and dashes. Additionally, MAAS *does* appear to support partial tag matches from my testing.

If somebody wants to use tags to achieve a similar result in the interim, the following is primitive and provided with no guarantees. It takes the simply takes the power address and replaces '.' with '_', creates the tag, then assigns it to the machine. There are likely better ways to do this but the following was certainly the easiest :)

```
#!/usr/bin/env python3

maas_user = 'replaceme'

import json
import subprocess
import sys

def create_tag(tag):
    try:
        subprocess.run(['maas', maas_user, 'tags', 'create',
            'name=' + tag], check=True, capture_output=True, text=True)
    except subprocess.CalledProcessError as e:
        sys.exit('ERROR: Failed to create tag "{}":\n{}'.format(tag, e.stderr))

def tag_machine(machine, tag):
    try:
        subprocess.run(['maas', maas_user, 'tag', 'update-nodes', tag, 'add='
            + machine], check=True, capture_output=True, text=True)
    except subprocess.CalledProcessError as e:
        sys.exit('ERROR: Failed to add tag "{}" to machine {}'.format(tag,
            machine))

def get_power_params():
    try:
        cmd = subprocess.run(['maas', maas_user, 'machines',
            'power-parameters'], check=True, capture_output=True, text=True)
    except subprocess.CalledProcessError as e:
        sys.exit('ERROR: Failed to get power-parameters:\n' + e.stderr)
    return json.loads(cmd.stdout)

def main():
    power_params = get_power_params()

    for machine in power_params:
        if 'power_address' in power_params[machine]:
            tag = power_params[machine]['power_address'].replace('.', '_')
            create_tag(tag)
            tag_machine(machine, tag)

if __name__ == '__main__':
    main()
```

[Alexsander de Souza (alexsander-souza)]

Feature request moved to Discourse: https://discourse.maas.io/t/filter-machines-by-power-address-serial-number-bmc-ilo-drac-etc/5264

[Thorsten Merten (thorsten-merten)]

The work on this issue has been done for the UI (see https://github.com/canonical/maas-ui/issues/2927). More for search and filter is being tracked in our internal product feature board as PF-3127 .