LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (https://www.linuxquestions.org/questions/programming-9/)
-   -   Parsing json blob with Python (https://www.linuxquestions.org/questions/programming-9/parsing-json-blob-with-python-4175714097/)

flukeyLinux 07-01-2022 08:34 AM

Parsing json blob with Python
 
Hello,
I'm having trouble finding much on this on the web. Any help would be massively appreciated.

I have the following json objects:

Code:

{
    "uuid": "gslbservice",
    "name": "toplevelurl.com",
    "groups": [
        {
            "name": "albpool1",
            "members": [
                {
                    "fqdn": "alb1.com",
                    "enabled": true
                }
            ],
        },
        {
            "name": "albpool2",
            "members": [
                {
                    "fqdn": "alb2.com",
                    "enabled": true
                }
            ],
        }
    ]
}

What I am trying to do is loop through all "members" objects (there could be 2,3 or 4) and updated the "enabled" key to "True" if for example the fqdn = "alb2.com."

I can do this with the first object with:
Code:

          route = "enable"
                fqdn = "alb1"
                for member in element['groups']:
                    if args.fqdn in member['members'][0]['fqdn']:
                        if args.route == "enable":
                            parsed['enabled'] = True

I think my problem is this line:
Code:

if args.fqdn in member['members'][0]['fqdn']
-- explicitly the [0].

is there a way to loop through all objects?

thanks in advance

pan64 07-01-2022 09:03 AM

yes, you replace that 0 with a loop, like:
Code:

for member in element['groups']:
    for data in member['members']:
        if args.fqdn in data['fqdn']:
            ....


flukeyLinux 07-01-2022 09:37 AM

Quote:

Originally Posted by pan64 (Post 6364882)
yes, you replace that 0 with a loop, like:
Code:

for member in element['groups']:
    for data in member['members']:
        if args.fqdn in data['fqdn']:
            ....


Lovely, so simple :)

thank you.

do you happen to know how to update a json key that is nested?

Code:

parsed['enabled'] = True
works.
But it is added at the wrong level.

Code:

parsed(member['enabled']) = True
Does not work.

this is my last hurdle

pan64 07-01-2022 09:41 AM

Quote:

Originally Posted by flukeyLinux (Post 6364884)
Lovely, so simple :)

thank you.

do you happen to know how to update a json key that is nested?

Code:

parsed['enabled'] = True
works.
But it is added at the wrong level.

Code:

parsed(member['enabled']) = True
Does not work.

this is my last hurdle

this is the same as here:
Code:

member['members'][0]['fqdn'] = new_value
(I have no idea what is that parsed

flukeyLinux 07-01-2022 09:54 AM

Quote:

Originally Posted by pan64 (Post 6364885)
this is the same as here:
Code:

member['members'][0]['fqdn'] = new_value
(I have no idea what is that parsed

parsed is basically the json data:

Code:

parsed = json.loads(json_data)
This does not work:
Code:

member['members'][0]['fqdn'] = new_value
parsed['members'][0]['enabled'] = False
KeyError: 'members'

messed about with a few variations of that with no joy.

boughtonp 07-01-2022 09:59 AM

Quote:

Originally Posted by flukeyLinux (Post 6364886)
parsed['members'][0]['enabled'] = False
KeyError: 'members'

The error message is clear - there is a key error with 'members' - i.e. there is no key with that name.

This is equally evident from merely reading the JSON snippet in post #1.


sundialsvcs 07-01-2022 10:03 AM

Just grab the JSON, decode it, and print the resulting structure.

The correct way to handle JSON is always to first decode it, then manipulate the resulting data structure, then encode it. Treat the JSON conversion process as a "black box." Python, like every other language these days, uses "known good" software to do this. Do not attempt to manipulate the JSON data textually.

flukeyLinux 07-01-2022 10:29 AM

Quote:

Originally Posted by boughtonp (Post 6364887)
The error message is clear - there is a key error with 'members' - i.e. there is no key with that name.

This is equally evident from merely reading the JSON snippet in post #1.


As you can "clearly" see (just to be as condescending as you :) ) from my first comment. the json blob contains object = "members".

Hence why I am asking the question...

flukeyLinux 07-01-2022 10:30 AM

Quote:

Originally Posted by sundialsvcs (Post 6364888)
Just grab the JSON, decode it, and print the resulting structure.

The correct way to handle JSON is always to first decode it, then manipulate the resulting data structure, then encode it. Treat the JSON conversion process as a "black box." Python, like every other language these days, uses "known good" software to do this. Do not attempt to manipulate the JSON data textually.

This information is good. I am new enough to python and have not come across decoding the json first. I must look that up. I am more of a bash head and this would be very easy there. But I need to use python as its a lambda.

boughtonp 07-01-2022 10:45 AM

Quote:

Originally Posted by flukeyLinux (Post 6364898)
...the json blob contains "members".

Hence why I am asking the question...

You need to back-up and learn basic JSON before going any further.


flukeyLinux 07-01-2022 10:54 AM

Quote:

Originally Posted by boughtonp (Post 6364907)
You need to back-up and learn basic JSON before going any further.


Good stuff. thanks lad. I had thought this forum was all part of it. good to know though.... :rolleyes:

dugan 07-01-2022 11:10 AM

Well, first, JSON in your code sample isn't valid. You can see that by pasting it into https://jsonlint.com/ ...

Also, I'm not sure if you know how to iterate through a dictionary. Looping through it in a "for in" loop will get you its keys. Usually you want to call its .keys() (remember, explicit is better than implicit), .values() or .items() method and loop through that, depending on whether you need its keys, values, or both.

ondoho 07-02-2022 02:35 AM

As a side remark I want to add that in Linux-speak "blob" means an opaque piece of data (a file), something you cannot (easily) read, e.g. a binary closed source blob, some firmware etc.

JSON is not that.
It is a simple text format, you can look at it in a text editor etc.

There's no need to "decode" it, just format it nicely.
What python does, it loads it into a structure that makes it easier to work with.
There are other (command line) tools that can do similar things, jq comes to mind.

ntubski 07-02-2022 09:07 AM

Quote:

Originally Posted by flukeyLinux (Post 6364877)
What I am trying to do is loop through all "members" objects (there could be 2,3 or 4) and updated the "enabled" key to "True" if for example the fqdn = "alb2.com."

This is slightly confusing because it's already "true" in your example input?

Quote:

Originally Posted by flukeyLinux (Post 6364886)
parsed is basically the json data:

Code:

parsed = json.loads(json_data)

Assuming element is somehow a subpart of parsed, I'm guessing you want

Code:

for member in element['groups']:
    for data in member['members']:
        if args.fqdn in data['fqdn']: # should this be "=" instead of "in"?
            data['enabled'] = True


Quote:

Originally Posted by flukeyLinux (Post 6364899)
I am new enough to python and have not come across decoding the json first.

Your parsed = json.loads(json_data) step is decoding.

Quote:

Originally Posted by ondoho (Post 6365032)
As a side remark I want to add that in Linux-speak "blob" means an opaque piece of data (a file)[...]
There's no need to "decode" it, just format it nicely.

I think sundialscvs is just using non-standard terminology, as usual (in this case, "decode" = "parse", "blob" = "unparsed data").


Quote:

Originally Posted by dugan (Post 6364912)
Also, I'm not sure if you know how to iterate through a dictionary.

They're iterating over the arrays, not the dictionaries, I think? (I'm assuming that the JSON array gets parsed into a Python array.)

sundialsvcs 07-02-2022 10:17 AM

@ondoho: What I am specifically saying is that you should treat the JSON data "as a blob." Don't try to "format it" – it doesn't need to be pretty – and don't try to write it yourself. Every programming language has encode/decode support for JSON (and other formats), and uses a standard set of binaries to do it. In this way you know that you can do what needs be done – to interchange structured data reliably with another client. You're handed a "blob" and the means to turn it into a data structure; then, to turn a data structure back into another "blob." And, if you do it this way, you can be confident that it will work correctly.


All times are GMT -5. The time now is 02:19 PM.