convert ceilometer output to python dataframe

Question

I'm a little new to Python and openstack ceilometer. I'm reading ceilometer data using the following code:

import ceilometerclient.client

cclient = ceilometerclient.client.get_client(2, os_username="Ceilometeradmin", os_password="blahblah", os_tenant_name="blahblah", os_auth_url="http://xxx.xx.xx.x:5000/v2.0")

query = [dict(field='resource_id', op='eq', value='dd893564-85e5-43f8-a384-086417f1d82c')]

ls = cclient.meters.list(q=query)

(Please see picture of output attached)

Does anyone know how i could convert this into a dataframe?

I tried : ls2 = pandas.DataFrame(ls, columns=["user_id", "name", "resource_id", "source", "meter_id", "project_id", "type", "unit"])

but get the following error:

Traceback (most recent call last):
File "", line 1, in File "/usr/lib/python2.7/dist-packages/pandas/core/frame.py", line 250, in init copy=copy)
File "/usr/lib/python2.7/dist-packages/pandas/core/frame.py", line 363, in _init_ndarray return create_block_manager_from_blocks([values.T], [columns, index])
File "/usr/lib/python2.7/dist-packages/pandas/core/internals.py", line 3750, in create_block_manager_from_blocks construction_error(tot_items, blocks[0].shape[1:], axes, e)
File "/usr/lib/python2.7/dist-packages/pandas/core/internals.py", line 3732, in construction_error passed,implied))
ValueError: Shape of passed values is (1, 2), indices imply (8, 2)

if someone could help it would really be much much appreciated..

Thank you

Best Wishes T

Answer 1

With the help of a colleague we managed to find the solution. I'm posting it in case it's useful to anyone trying to convert ceilometer data into a dataframe.

ls2 = str(ls)[8:-2]
tmp = ls2.replace("u'","'")
tmp = tmp.replace("<Meter","")
tmp = tmp.replace("}>","}")
tmp= "["+tmp+"]"
tmp = tmp.replace("'", "\"")
parsed = json.loads(tmp)
ls3 = pandas.DataFrame(parsed)

Replaced characters which make it as invalid dictionary and converted into dataframe