)]}'
{"specs/liberty/Improve-instance-metering.rst":[{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"7bdda62144e3d29fb0dcbc6b74e31c48347ce0f9","unresolved":false,"context_lines":[{"line_number":33,"context_line":""},{"line_number":34,"context_line":"To dig the performance singularity of the compute-agent process, for the"},{"line_number":35,"context_line":"proposal[1] assuming we can deploy 100 instances on one host at most. we can"},{"line_number":36,"context_line":"find it is not a very significant improvement reducing 100 instances to 20"},{"line_number":37,"context_line":"instances of every Nova list query. But it will be a significant improvement"},{"line_number":38,"context_line":"that reducing the times of Nova API requests from 500 requests/min to"},{"line_number":39,"context_line":"1 requests/min, this is the goal of this proposal."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_c464d94a","line":36,"updated":"2015-08-10 12:55:05.000000000","message":"what happens if you\u0027re reducing from 100 to 1 (or 0)?  the idea of changes-since is that unless you are creating/deleting instances rapidly, most instances will not change that frequently so you will often not get any change between polling cycles.","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"f1e6a54e76aa183477d56ae64eb7a7256e3cb45d","unresolved":false,"context_lines":[{"line_number":33,"context_line":""},{"line_number":34,"context_line":"To dig the performance singularity of the compute-agent process, for the"},{"line_number":35,"context_line":"proposal[1] assuming we can deploy 100 instances on one host at most. we can"},{"line_number":36,"context_line":"find it is not a very significant improvement reducing 100 instances to 20"},{"line_number":37,"context_line":"instances of every Nova list query. But it will be a significant improvement"},{"line_number":38,"context_line":"that reducing the times of Nova API requests from 500 requests/min to"},{"line_number":39,"context_line":"1 requests/min, this is the goal of this proposal."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_998953cd","line":36,"in_reply_to":"1a4dcd0f_c464d94a","updated":"2015-08-10 14:19:03.000000000","message":"yeah, the metadata caching proposal aims to reduce the amount of the return every polling nova API, but don\u0027t reduce the polling of virt api (inspector), it use the metadata cached in memory, pollsters will merged the resource (instances) metadata and metrics from inspector to *samples* and publish them to storage finally. but before recording the samples to storage (gnocchi and other drivers), the resource metadata will be extract from samples and recorded to *resource* table. and the samples table will only record the metric data. the resource metadata is unique by resource_id, that is to say, if the resource metadata didn\u0027t changed (I mean the instance object from nova layer) it shouldn\u0027t be polling and updated.\n\nthe change-since can also be used if we move the instance metering to central-agent, most instances will don\u0027t be query. that can also reduce the amount of query result.","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"563405b436fc3a6cf7e0627dc8e9d3b9239a3f86","unresolved":false,"context_lines":[{"line_number":55,"context_line":"  will use \"instance_get_all\" to query all instance from Nova API and generate"},{"line_number":56,"context_line":"  samples \"instance\". The \"instance_get_all\" has also applied \"change-since\""},{"line_number":57,"context_line":"  parameter as query filter to massively reduce the amount of every query and"},{"line_number":58,"context_line":"  cache the instance info didn\u0027t updated. see metadata-caching proposal [1]."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"* The compute-agent running on compute nodes will collector instance metrics"},{"line_number":61,"context_line":"  only from virt and don\u0027t need the instance info from nova any more."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_bb15b140","line":58,"updated":"2015-08-06 12:01:09.000000000","message":"the instance meter will be deleted after this cycle.","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"f1e6a54e76aa183477d56ae64eb7a7256e3cb45d","unresolved":false,"context_lines":[{"line_number":55,"context_line":"  will use \"instance_get_all\" to query all instance from Nova API and generate"},{"line_number":56,"context_line":"  samples \"instance\". The \"instance_get_all\" has also applied \"change-since\""},{"line_number":57,"context_line":"  parameter as query filter to massively reduce the amount of every query and"},{"line_number":58,"context_line":"  cache the instance info didn\u0027t updated. see metadata-caching proposal [1]."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"* The compute-agent running on compute nodes will collector instance metrics"},{"line_number":61,"context_line":"  only from virt and don\u0027t need the instance info from nova any more."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_5cfc193e","line":58,"in_reply_to":"1a4dcd0f_44e4490c","updated":"2015-08-10 14:19:03.000000000","message":"if so, how can we query the statistics based on instances amount ? as I know, there is use case that need to get the amount of instance by *count* aggregation method on *instance* metrics.","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"041afa0ce74c79ac8fe3ae2c3bac6875b1a0f92a","unresolved":false,"context_lines":[{"line_number":55,"context_line":"  will use \"instance_get_all\" to query all instance from Nova API and generate"},{"line_number":56,"context_line":"  samples \"instance\". The \"instance_get_all\" has also applied \"change-since\""},{"line_number":57,"context_line":"  parameter as query filter to massively reduce the amount of every query and"},{"line_number":58,"context_line":"  cache the instance info didn\u0027t updated. see metadata-caching proposal [1]."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"* The compute-agent running on compute nodes will collector instance metrics"},{"line_number":61,"context_line":"  only from virt and don\u0027t need the instance info from nova any more."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_bc72f87a","line":58,"in_reply_to":"1a4dcd0f_5cfc193e","updated":"2015-08-10 15:09:56.000000000","message":"the following is the instance meters, what is the count value used here? it only represents the number of times you\u0027ve polled.\n\n  +--------+----------------------------+----------------------------+-----+-----+-----+--------+-------+-------------+----------------------------+----------------------------+\n| Period | Period Start               | Period End                 | Max | Min | Avg | Sum    | Count | Duration    | Duration Start             | Duration End               |\n+--------+----------------------------+----------------------------+-----+-----+-----+--------+-------+-------------+----------------------------+----------------------------+\n| 0      | 2015-08-10T14:30:20.311422 | 2015-08-10T15:00:18.314857 | 1.0 | 1.0 | 1.0 | 2080.0 | 2080  | 1798.003435 | 2015-08-10T14:30:20.311422 | 2015-08-10T15:00:18.314857 |\n+--------+----------------------------+----------------------------+-----+-----+-----+--------+-------+-------------+----------------------------+----------------------------+","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"7bdda62144e3d29fb0dcbc6b74e31c48347ce0f9","unresolved":false,"context_lines":[{"line_number":55,"context_line":"  will use \"instance_get_all\" to query all instance from Nova API and generate"},{"line_number":56,"context_line":"  samples \"instance\". The \"instance_get_all\" has also applied \"change-since\""},{"line_number":57,"context_line":"  parameter as query filter to massively reduce the amount of every query and"},{"line_number":58,"context_line":"  cache the instance info didn\u0027t updated. see metadata-caching proposal [1]."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"* The compute-agent running on compute nodes will collector instance metrics"},{"line_number":61,"context_line":"  only from virt and don\u0027t need the instance info from nova any more."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_44e4490c","line":58,"in_reply_to":"1a4dcd0f_76e50036","updated":"2015-08-10 12:55:05.000000000","message":"no. any meter that has volume\u003d1 hardcoded will be dropped. i\u0027m not sure if you are referencing the instance meter specifically or the \"meters related to instance\". if the first, it\u0027s being dropped.","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"3ff99895588808b1653ca310b86c13e849a10c25","unresolved":false,"context_lines":[{"line_number":55,"context_line":"  will use \"instance_get_all\" to query all instance from Nova API and generate"},{"line_number":56,"context_line":"  samples \"instance\". The \"instance_get_all\" has also applied \"change-since\""},{"line_number":57,"context_line":"  parameter as query filter to massively reduce the amount of every query and"},{"line_number":58,"context_line":"  cache the instance info didn\u0027t updated. see metadata-caching proposal [1]."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"* The compute-agent running on compute nodes will collector instance metrics"},{"line_number":61,"context_line":"  only from virt and don\u0027t need the instance info from nova any more."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_76e50036","line":58,"in_reply_to":"1a4dcd0f_bb15b140","updated":"2015-08-06 13:00:38.000000000","message":"IIUC, the instance meter from notification instance.exist will be deleted, but the instance meter from compute-agent will be keeped, right?","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"a4d95ea273455e196789621fd313ea1336be3821","unresolved":false,"context_lines":[{"line_number":55,"context_line":"  will use \"instance_get_all\" to query all instance from Nova API and generate"},{"line_number":56,"context_line":"  samples \"instance\". The \"instance_get_all\" has also applied \"change-since\""},{"line_number":57,"context_line":"  parameter as query filter to massively reduce the amount of every query and"},{"line_number":58,"context_line":"  cache the instance info didn\u0027t updated. see metadata-caching proposal [1]."},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"* The compute-agent running on compute nodes will collector instance metrics"},{"line_number":61,"context_line":"  only from virt and don\u0027t need the instance info from nova any more."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_6aa55e02","line":58,"in_reply_to":"1a4dcd0f_bc72f87a","updated":"2015-08-12 01:10:11.000000000","message":"yeah, I just realized I was wrong, I thought if we provide start_timestamp and end_timestamp that the timespan lt a polling interval, the query can be used to count the amount of instances.\nthe idea is stupid ;(. If we have plan to remove *instance* of compute-agent. how about add a new metrics *instance* to central-agent to polling resource metadata only? or other metric name than \"instance\" ? if the proposal can be accepted. it is easy to implement that.","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"563405b436fc3a6cf7e0627dc8e9d3b9239a3f86","unresolved":false,"context_lines":[{"line_number":59,"context_line":""},{"line_number":60,"context_line":"* The compute-agent running on compute nodes will collector instance metrics"},{"line_number":61,"context_line":"  only from virt and don\u0027t need the instance info from nova any more."},{"line_number":62,"context_line":"  Typically, the nova created instance xml from libvirt include the basic info"},{"line_number":63,"context_line":"  we needed, the instance_uuid, the user_id, the project_id. For more"},{"line_number":64,"context_line":"  information, please see [2]. With this basic info and the measurements by"},{"line_number":65,"context_line":"  inspector querying from libvirt API, it is enough to generate the instance"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_9053f438","line":62,"updated":"2015-08-06 12:01:09.000000000","message":"is this a libvirt only change?","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"3ff99895588808b1653ca310b86c13e849a10c25","unresolved":false,"context_lines":[{"line_number":59,"context_line":""},{"line_number":60,"context_line":"* The compute-agent running on compute nodes will collector instance metrics"},{"line_number":61,"context_line":"  only from virt and don\u0027t need the instance info from nova any more."},{"line_number":62,"context_line":"  Typically, the nova created instance xml from libvirt include the basic info"},{"line_number":63,"context_line":"  we needed, the instance_uuid, the user_id, the project_id. For more"},{"line_number":64,"context_line":"  information, please see [2]. With this basic info and the measurements by"},{"line_number":65,"context_line":"  inspector querying from libvirt API, it is enough to generate the instance"}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_d6d094f8","line":62,"in_reply_to":"1a4dcd0f_9053f438","updated":"2015-08-06 13:00:38.000000000","message":"yeah, this is main thing I concerning about. we need to keep compatible with different virt drivers. I have only checked the libvirt driver. for other virt drivers if they don\u0027t have this info (that will need a dig), I have proposed a compromise solution that, the instance metrics will be collect and pushed to collector finally. before collector recording the samples to the storage, these info can be get from storage that the *instance* meter provided.\n\nDoes not necessarily if the heperv, xenapi and other virt drivers have these info in virt layer. I will read the virt driver implementation of Nova.","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"563405b436fc3a6cf7e0627dc8e9d3b9239a3f86","unresolved":false,"context_lines":[{"line_number":60,"context_line":"* The compute-agent running on compute nodes will collector instance metrics"},{"line_number":61,"context_line":"  only from virt and don\u0027t need the instance info from nova any more."},{"line_number":62,"context_line":"  Typically, the nova created instance xml from libvirt include the basic info"},{"line_number":63,"context_line":"  we needed, the instance_uuid, the user_id, the project_id. For more"},{"line_number":64,"context_line":"  information, please see [2]. With this basic info and the measurements by"},{"line_number":65,"context_line":"  inspector querying from libvirt API, it is enough to generate the instance"},{"line_number":66,"context_line":"  samples of disk, cpu, net, mem, etc."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_db951d87","line":63,"updated":"2015-08-06 12:01:09.000000000","message":"we generate metadata with each sample[1]. how will this be handled?\n\n[1] https://github.com/openstack/ceilometer/blob/master/ceilometer/nova_client.py#L88-L136","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"f1e6a54e76aa183477d56ae64eb7a7256e3cb45d","unresolved":false,"context_lines":[{"line_number":60,"context_line":"* The compute-agent running on compute nodes will collector instance metrics"},{"line_number":61,"context_line":"  only from virt and don\u0027t need the instance info from nova any more."},{"line_number":62,"context_line":"  Typically, the nova created instance xml from libvirt include the basic info"},{"line_number":63,"context_line":"  we needed, the instance_uuid, the user_id, the project_id. For more"},{"line_number":64,"context_line":"  information, please see [2]. With this basic info and the measurements by"},{"line_number":65,"context_line":"  inspector querying from libvirt API, it is enough to generate the instance"},{"line_number":66,"context_line":"  samples of disk, cpu, net, mem, etc."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_27b6dadc","line":63,"in_reply_to":"1a4dcd0f_76742035","updated":"2015-08-10 14:19:03.000000000","message":"I have found that we just need the resource_id (instance uuid), because the sample item in storage just need resource_id and metrics data, see[1]. the other info of instance from Nova layer, will be polled by central-agent. the resource_id, I have simply read the virt driver of nova code, the instance id will be written to instance configure file (like xml of libvirt). that will need further confirm.\n\n\n[1] https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/sqlalchemy/models.py#L203-L221","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"3ff99895588808b1653ca310b86c13e849a10c25","unresolved":false,"context_lines":[{"line_number":60,"context_line":"* The compute-agent running on compute nodes will collector instance metrics"},{"line_number":61,"context_line":"  only from virt and don\u0027t need the instance info from nova any more."},{"line_number":62,"context_line":"  Typically, the nova created instance xml from libvirt include the basic info"},{"line_number":63,"context_line":"  we needed, the instance_uuid, the user_id, the project_id. For more"},{"line_number":64,"context_line":"  information, please see [2]. With this basic info and the measurements by"},{"line_number":65,"context_line":"  inspector querying from libvirt API, it is enough to generate the instance"},{"line_number":66,"context_line":"  samples of disk, cpu, net, mem, etc."}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_76742035","line":63,"in_reply_to":"1a4dcd0f_db951d87","updated":"2015-08-06 13:00:38.000000000","message":"yeah, the resource metatada will be generated by central-agent by *instance* meters, for these instance metrics, they have same resource id with the *instance* meters, so the resource metadata will be shared. for using Gnocchi, the situation is similar.","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"3c465667520d5bee00aa3c83d10b81fe51bbab7a","unresolved":false,"context_lines":[{"line_number":63,"context_line":"  we needed, the instance_uuid, the user_id, the project_id. For more"},{"line_number":64,"context_line":"  information, please see [2]. With this basic info and the measurements by"},{"line_number":65,"context_line":"  inspector querying from libvirt API, it is enough to generate the instance"},{"line_number":66,"context_line":"  samples of disk, cpu, net, mem, etc."},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"Additionally, there maybe need some other changes:"},{"line_number":69,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_d54ace25","line":66,"updated":"2015-08-06 11:27:29.000000000","message":"Is there always correspondence between the instance uuid and nova\u0027s resource_id?\n\nWhat about VMs running under libvirt but not being managed by Nova? Is that something to be concerned with?","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"3ff99895588808b1653ca310b86c13e849a10c25","unresolved":false,"context_lines":[{"line_number":63,"context_line":"  we needed, the instance_uuid, the user_id, the project_id. For more"},{"line_number":64,"context_line":"  information, please see [2]. With this basic info and the measurements by"},{"line_number":65,"context_line":"  inspector querying from libvirt API, it is enough to generate the instance"},{"line_number":66,"context_line":"  samples of disk, cpu, net, mem, etc."},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"Additionally, there maybe need some other changes:"},{"line_number":69,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a4dcd0f_1b0ce539","line":66,"in_reply_to":"1a4dcd0f_d54ace25","updated":"2015-08-06 13:00:38.000000000","message":"1. As replied for gordon L62, for libvirt driver, the instance uuid from Nova is definitely written to instnace xml. and I need a deeper dig for others virt drivers implemented in Nova.\n\n2. you mean orphan instances ? I have described this issue following, in a typical openstack deployment, I don\u0027t think is reasonable that there are may orphan instances. and we can filter and drop the orphan instances before recording the samples.","commit_id":"9babc91e83d9201e5ad05551aac232c595b857a1"},{"author":{"_account_id":1669,"name":"Julien Danjou","display_name":"jd","email":"julien@danjou.info","username":"jdanjou"},"change_message_id":"5d9e24bb5ed1f504b7853b2ecfa7ad95f21c6e47","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"This proposal will avoid every compute-agents polling Nova API periodically. In"},{"line_number":14,"context_line":"production scenario, this change will greatly reduce the workload of Nova api"},{"line_number":15,"context_line":"caused by compute-agents polling."},{"line_number":16,"context_line":""},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"Problem description"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_993e85b6","line":15,"updated":"2015-08-13 11:14:57.000000000","message":"Can we stop talking about compute-agents which does not exist anymore?","commit_id":"3fb5766eee09729074cf687c067a935d277ad47d"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"cc21d2402828316fddc4e88278e8fa04fae290b2","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"This proposal will avoid every compute-agents polling Nova API periodically. In"},{"line_number":14,"context_line":"production scenario, this change will greatly reduce the workload of Nova api"},{"line_number":15,"context_line":"caused by compute-agents polling."},{"line_number":16,"context_line":""},{"line_number":17,"context_line":""},{"line_number":18,"context_line":"Problem description"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_c235d2db","line":15,"in_reply_to":"1a4dcd0f_993e85b6","updated":"2015-08-14 01:14:02.000000000","message":"my mistake, this spec just to propose this idea and want to known you opinion and suggestions ;). I am not sure we can accept new spec now, because L2 release. If this is make sense for you, will try to implement in future.","commit_id":"3fb5766eee09729074cf687c067a935d277ad47d"},{"author":{"_account_id":1669,"name":"Julien Danjou","display_name":"jd","email":"julien@danjou.info","username":"jdanjou"},"change_message_id":"5d9e24bb5ed1f504b7853b2ecfa7ad95f21c6e47","unresolved":false,"context_lines":[{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* Move the *instance* metric from compute-agent to central-agent. central-agent"},{"line_number":55,"context_line":"  will use \"instance_get_all\" to query all instance from Nova API and generate"},{"line_number":56,"context_line":"  samples \"instance\". The \"instance_get_all\" has also applied \"change-since\""},{"line_number":57,"context_line":"  parameter as query filter to massively reduce the amount of every query and"},{"line_number":58,"context_line":"  cache the instance info didn\u0027t updated. see metadata-caching proposal [1]."},{"line_number":59,"context_line":"  The *instance* meter is used for collecting resource metadata, the resource"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_1987d561","line":56,"updated":"2015-08-13 11:14:57.000000000","message":"This is not a change of agents (which does not exists anymore) but of namespace rather.","commit_id":"3fb5766eee09729074cf687c067a935d277ad47d"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"cc21d2402828316fddc4e88278e8fa04fae290b2","unresolved":false,"context_lines":[{"line_number":53,"context_line":""},{"line_number":54,"context_line":"* Move the *instance* metric from compute-agent to central-agent. central-agent"},{"line_number":55,"context_line":"  will use \"instance_get_all\" to query all instance from Nova API and generate"},{"line_number":56,"context_line":"  samples \"instance\". The \"instance_get_all\" has also applied \"change-since\""},{"line_number":57,"context_line":"  parameter as query filter to massively reduce the amount of every query and"},{"line_number":58,"context_line":"  cache the instance info didn\u0027t updated. see metadata-caching proposal [1]."},{"line_number":59,"context_line":"  The *instance* meter is used for collecting resource metadata, the resource"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_224b5661","line":56,"in_reply_to":"1a4dcd0f_1987d561","updated":"2015-08-14 01:14:02.000000000","message":"thanks, will be done.","commit_id":"3fb5766eee09729074cf687c067a935d277ad47d"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"a5d383f5049096d10c7942b92dc1ffe557617594","unresolved":false,"context_lines":[{"line_number":58,"context_line":"  cache the instance info didn\u0027t updated. see metadata-caching proposal [1]."},{"line_number":59,"context_line":"  The *instance* meter is used for collecting resource metadata, the resource"},{"line_number":60,"context_line":"  metadata here means instance info from nova db, and only polling the"},{"line_number":61,"context_line":"  instances updated."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"* The compute-agent running on compute nodes will collect instance metrics"},{"line_number":64,"context_line":"  only from virt layer and don\u0027t need the instance info from nova any more."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_9671ac0e","line":61,"updated":"2015-08-12 12:24:15.000000000","message":"i don\u0027t think instance metric will be a good idea. the whole idea of disabling volume\u003d1 metrics is to avoid storing non-measurement data as meters/metrics.\n\ntbh, i think the resource metadata captured in gnocchi should be done in events and the two data correlated that way. what if you have an event that captures active resources and metadata? it would be a much similar query as well.","commit_id":"3fb5766eee09729074cf687c067a935d277ad47d"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"c37be3dab2b949a032750a81421f6e812709034a","unresolved":false,"context_lines":[{"line_number":58,"context_line":"  cache the instance info didn\u0027t updated. see metadata-caching proposal [1]."},{"line_number":59,"context_line":"  The *instance* meter is used for collecting resource metadata, the resource"},{"line_number":60,"context_line":"  metadata here means instance info from nova db, and only polling the"},{"line_number":61,"context_line":"  instances updated."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"* The compute-agent running on compute nodes will collect instance metrics"},{"line_number":64,"context_line":"  only from virt layer and don\u0027t need the instance info from nova any more."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_d90e803c","line":61,"in_reply_to":"1a4dcd0f_00cc71b9","updated":"2015-08-12 14:11:46.000000000","message":"polled data are/can be captured as events too. meters/metrics are just a subset of event data","commit_id":"3fb5766eee09729074cf687c067a935d277ad47d"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"ec67e4378c0cedd1c0e784a226afdb620b7ae90d","unresolved":false,"context_lines":[{"line_number":58,"context_line":"  cache the instance info didn\u0027t updated. see metadata-caching proposal [1]."},{"line_number":59,"context_line":"  The *instance* meter is used for collecting resource metadata, the resource"},{"line_number":60,"context_line":"  metadata here means instance info from nova db, and only polling the"},{"line_number":61,"context_line":"  instances updated."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"* The compute-agent running on compute nodes will collect instance metrics"},{"line_number":64,"context_line":"  only from virt layer and don\u0027t need the instance info from nova any more."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_00cc71b9","line":61,"in_reply_to":"1a4dcd0f_9671ac0e","updated":"2015-08-12 13:18:25.000000000","message":"Using events to update resource metadata in ceilometer or Gnocchi will be more efficient because the resource metadata may only need to be update when something about resource happened. but I have some concerns about that approach:\n\n* the resource netadata updating will depend on events collection, what if some events didn\u0027t be collected ?\n* we need to ensure all the resource update actions in other projects of OpenStack have corresponding events published, that need other resources providers to ensure.\n* Does the events include enough info for ceilometer? and all the events include enough info we needed. we may need the resource metadata do some filter query. e.g. use query filtered by instance metadata do auto-scaling.","commit_id":"3fb5766eee09729074cf687c067a935d277ad47d"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"f353826686e73b3a5dba77cb7f215cef8ef678ee","unresolved":false,"context_lines":[{"line_number":58,"context_line":"  cache the instance info didn\u0027t updated. see metadata-caching proposal [1]."},{"line_number":59,"context_line":"  The *instance* meter is used for collecting resource metadata, the resource"},{"line_number":60,"context_line":"  metadata here means instance info from nova db, and only polling the"},{"line_number":61,"context_line":"  instances updated."},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"* The compute-agent running on compute nodes will collect instance metrics"},{"line_number":64,"context_line":"  only from virt layer and don\u0027t need the instance info from nova any more."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_01de97dc","line":61,"in_reply_to":"1a4dcd0f_d90e803c","updated":"2015-08-13 01:57:25.000000000","message":"you mean meters published through notification-agent ? but that will also need the polled data of instances from agent if my above concerns really exist.","commit_id":"3fb5766eee09729074cf687c067a935d277ad47d"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"a5d383f5049096d10c7942b92dc1ffe557617594","unresolved":false,"context_lines":[{"line_number":86,"context_line":"collected by central-agent will can be associated with samples by unique"},{"line_number":87,"context_line":"\"resource_id\", the resource_id can be directly get from virt driver. these"},{"line_number":88,"context_line":"changes will avoid frequent Nova API requests and make the instance related"},{"line_number":89,"context_line":"samples more lightweight."},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"FYI: there is an old proposal for compute-agent improvement tried to do similar"},{"line_number":92,"context_line":"works, but is different, see [3]."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_96144ce9","line":89,"updated":"2015-08-12 12:24:15.000000000","message":"this should probably target M* cycle. while this makes sense, one reason we couldn\u0027t do this earlier is because our model doesn\u0027t separate metric and metadata -- they are tightly coupled.\n\nwith gnocchi, it\u0027s less coupled and this might be valid but right now if we implement it, we the metadata attached to metrics may all be stale.","commit_id":"3fb5766eee09729074cf687c067a935d277ad47d"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"c37be3dab2b949a032750a81421f6e812709034a","unresolved":false,"context_lines":[{"line_number":86,"context_line":"collected by central-agent will can be associated with samples by unique"},{"line_number":87,"context_line":"\"resource_id\", the resource_id can be directly get from virt driver. these"},{"line_number":88,"context_line":"changes will avoid frequent Nova API requests and make the instance related"},{"line_number":89,"context_line":"samples more lightweight."},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"FYI: there is an old proposal for compute-agent improvement tried to do similar"},{"line_number":92,"context_line":"works, but is different, see [3]."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_94699fc0","line":89,"in_reply_to":"1a4dcd0f_20bded67","updated":"2015-08-12 14:11:46.000000000","message":"yes, that is how i modelled the sql backend which is probably why it\u0027s a lot more efficient than mongodb in terms of db size. but that said, when we return results, resource and metric data are combined together again which is the problem.\n\nif we use this solution, the metadata and metric we combine as result won\u0027t be accurate, the metadata will always be stale (depending on how far apart the compute and central polls happen)","commit_id":"3fb5766eee09729074cf687c067a935d277ad47d"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"f353826686e73b3a5dba77cb7f215cef8ef678ee","unresolved":false,"context_lines":[{"line_number":86,"context_line":"collected by central-agent will can be associated with samples by unique"},{"line_number":87,"context_line":"\"resource_id\", the resource_id can be directly get from virt driver. these"},{"line_number":88,"context_line":"changes will avoid frequent Nova API requests and make the instance related"},{"line_number":89,"context_line":"samples more lightweight."},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"FYI: there is an old proposal for compute-agent improvement tried to do similar"},{"line_number":92,"context_line":"works, but is different, see [3]."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_14a1f393","line":89,"in_reply_to":"1a4dcd0f_94699fc0","updated":"2015-08-13 01:57:25.000000000","message":"yes, this is a problem, I thinks the problem is only the real-time of resource metadata. for metrics of a resource, we even only care  \"meter_name\" and \"volume\", but we need metrics associated with resource metadata to perform filter query.\n\nIMO, all the polling solutions to collect data are not accurate and have delay. that depends on the acceptance of users. in production scenario, the polling interval won\u0027t be 10 min, even 1 min they have complained it is too long, but a shorter period will lead performance issue...","commit_id":"3fb5766eee09729074cf687c067a935d277ad47d"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"ec67e4378c0cedd1c0e784a226afdb620b7ae90d","unresolved":false,"context_lines":[{"line_number":86,"context_line":"collected by central-agent will can be associated with samples by unique"},{"line_number":87,"context_line":"\"resource_id\", the resource_id can be directly get from virt driver. these"},{"line_number":88,"context_line":"changes will avoid frequent Nova API requests and make the instance related"},{"line_number":89,"context_line":"samples more lightweight."},{"line_number":90,"context_line":""},{"line_number":91,"context_line":"FYI: there is an old proposal for compute-agent improvement tried to do similar"},{"line_number":92,"context_line":"works, but is different, see [3]."}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a4dcd0f_20bded67","line":89,"in_reply_to":"1a4dcd0f_96144ce9","updated":"2015-08-12 13:18:25.000000000","message":"Yeah, I just drafted my idea now and want to know you opinions to see if this is idea is valuable :)\n\nGnocchi has definitely separated the resource metadata and metrics. For ceilometer storage backend, even the resource metadata and metrics are coupled when publishing, but it will be separated when recorded to storage ( have found sql backend) to separated tables, see[1], and they are associated with \"resource_id\", not sure I am missing something ;)\n\n[1] https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/sqlalchemy/models.py#L203-L221","commit_id":"3fb5766eee09729074cf687c067a935d277ad47d"}],"specs/mitaka/Improve-instance-metering.rst":[{"author":{"_account_id":7478,"name":"Nadya Shakhat","email":"nadmi4@gmail.com","username":"nprivalova"},"change_message_id":"0767a60e181c1814760bc5d93c3df1636d815169","unresolved":false,"context_lines":[{"line_number":9,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":10,"context_line":""},{"line_number":11,"context_line":"https://blueprints.launchpad.net/ceilometer/+spec/improve-nova-instance-metering"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"This proposal will avoid every ceilometer-polling with \"compute\" namespace"},{"line_number":14,"context_line":"polling Nova API periodically. In production scenario, this change will greatly"},{"line_number":15,"context_line":"reduce the workload of Nova api caused by ceilometer-polling polling."}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_43e72e12","line":12,"updated":"2015-09-10 09:07:55.000000000","message":"It\u0027s better to say \u0027This proposal is about...\u0027 or \u0027Now we have a great load on nova because .... This proposal aims to reduce the load by...\u0027","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"5ead137180a3ead053cf4160029a79510099dd93","unresolved":false,"context_lines":[{"line_number":9,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":10,"context_line":""},{"line_number":11,"context_line":"https://blueprints.launchpad.net/ceilometer/+spec/improve-nova-instance-metering"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"This proposal will avoid every ceilometer-polling with \"compute\" namespace"},{"line_number":14,"context_line":"polling Nova API periodically. In production scenario, this change will greatly"},{"line_number":15,"context_line":"reduce the workload of Nova api caused by ceilometer-polling polling."}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_ae7b81f3","line":12,"in_reply_to":"da20952f_43e72e12","updated":"2015-09-14 09:51:57.000000000","message":"Done","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":7478,"name":"Nadya Shakhat","email":"nadmi4@gmail.com","username":"nprivalova"},"change_message_id":"0767a60e181c1814760bc5d93c3df1636d815169","unresolved":false,"context_lines":[{"line_number":11,"context_line":"https://blueprints.launchpad.net/ceilometer/+spec/improve-nova-instance-metering"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"This proposal will avoid every ceilometer-polling with \"compute\" namespace"},{"line_number":14,"context_line":"polling Nova API periodically. In production scenario, this change will greatly"},{"line_number":15,"context_line":"reduce the workload of Nova api caused by ceilometer-polling polling."},{"line_number":16,"context_line":""},{"line_number":17,"context_line":"NOTE: The ceilometer-polling service with \"compute\" namespace is previously"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_a3b64af7","line":14,"updated":"2015-09-10 09:07:55.000000000","message":"one \u0027polling\u0027 may be removed","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"5ead137180a3ead053cf4160029a79510099dd93","unresolved":false,"context_lines":[{"line_number":11,"context_line":"https://blueprints.launchpad.net/ceilometer/+spec/improve-nova-instance-metering"},{"line_number":12,"context_line":""},{"line_number":13,"context_line":"This proposal will avoid every ceilometer-polling with \"compute\" namespace"},{"line_number":14,"context_line":"polling Nova API periodically. In production scenario, this change will greatly"},{"line_number":15,"context_line":"reduce the workload of Nova api caused by ceilometer-polling polling."},{"line_number":16,"context_line":""},{"line_number":17,"context_line":"NOTE: The ceilometer-polling service with \"compute\" namespace is previously"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_8e78fdf9","line":14,"in_reply_to":"da20952f_a3b64af7","updated":"2015-09-14 09:51:57.000000000","message":"Done","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":7478,"name":"Nadya Shakhat","email":"nadmi4@gmail.com","username":"nprivalova"},"change_message_id":"0767a60e181c1814760bc5d93c3df1636d815169","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"Problem description"},{"line_number":21,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":22,"context_line":""},{"line_number":23,"context_line":"To describe the problem, We assume we have a environment with 500 compute nodes"},{"line_number":24,"context_line":"and every compute node deployed with ceilometer-polling with \"compute\""},{"line_number":25,"context_line":"namespace (this is a real deployment scale of our company). the polling"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_23a8fa76","line":22,"updated":"2015-09-10 09:07:55.000000000","message":"consider the following example. There is an environment...","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"5ead137180a3ead053cf4160029a79510099dd93","unresolved":false,"context_lines":[{"line_number":19,"context_line":""},{"line_number":20,"context_line":"Problem description"},{"line_number":21,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":22,"context_line":""},{"line_number":23,"context_line":"To describe the problem, We assume we have a environment with 500 compute nodes"},{"line_number":24,"context_line":"and every compute node deployed with ceilometer-polling with \"compute\""},{"line_number":25,"context_line":"namespace (this is a real deployment scale of our company). the polling"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_2e21b1fc","line":22,"in_reply_to":"da20952f_23a8fa76","updated":"2015-09-14 09:51:57.000000000","message":"Done","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":7478,"name":"Nadya Shakhat","email":"nadmi4@gmail.com","username":"nprivalova"},"change_message_id":"0767a60e181c1814760bc5d93c3df1636d815169","unresolved":false,"context_lines":[{"line_number":22,"context_line":""},{"line_number":23,"context_line":"To describe the problem, We assume we have a environment with 500 compute nodes"},{"line_number":24,"context_line":"and every compute node deployed with ceilometer-polling with \"compute\""},{"line_number":25,"context_line":"namespace (this is a real deployment scale of our company). the polling"},{"line_number":26,"context_line":"interval is set as 60s. For every polling period, ceilometer-polling will"},{"line_number":27,"context_line":"invoke \"instance_get_all_by_host()\" method (and maybe need querying image and"},{"line_number":28,"context_line":"flavor also) to query the instances of this host from Nova API. That means"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_43cace50","line":25,"updated":"2015-09-10 09:07:55.000000000","message":"as -\u003e to","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"5ead137180a3ead053cf4160029a79510099dd93","unresolved":false,"context_lines":[{"line_number":22,"context_line":""},{"line_number":23,"context_line":"To describe the problem, We assume we have a environment with 500 compute nodes"},{"line_number":24,"context_line":"and every compute node deployed with ceilometer-polling with \"compute\""},{"line_number":25,"context_line":"namespace (this is a real deployment scale of our company). the polling"},{"line_number":26,"context_line":"interval is set as 60s. For every polling period, ceilometer-polling will"},{"line_number":27,"context_line":"invoke \"instance_get_all_by_host()\" method (and maybe need querying image and"},{"line_number":28,"context_line":"flavor also) to query the instances of this host from Nova API. That means"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_0e262d16","line":25,"in_reply_to":"da20952f_43cace50","updated":"2015-09-14 09:51:57.000000000","message":"Done","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":7478,"name":"Nadya Shakhat","email":"nadmi4@gmail.com","username":"nprivalova"},"change_message_id":"0767a60e181c1814760bc5d93c3df1636d815169","unresolved":false,"context_lines":[{"line_number":26,"context_line":"interval is set as 60s. For every polling period, ceilometer-polling will"},{"line_number":27,"context_line":"invoke \"instance_get_all_by_host()\" method (and maybe need querying image and"},{"line_number":28,"context_line":"flavor also) to query the instances of this host from Nova API. That means"},{"line_number":29,"context_line":"the Nova API will receive 500 requests meanwhile every minutes. It would"},{"line_number":30,"context_line":"massively increase the workload of Nova API."},{"line_number":31,"context_line":""},{"line_number":32,"context_line":"In the metadata caching proposal[1], the instance info from Nova will be cached"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_e3b8c2c9","line":29,"updated":"2015-09-10 09:07:55.000000000","message":"of -\u003e on","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"5ead137180a3ead053cf4160029a79510099dd93","unresolved":false,"context_lines":[{"line_number":26,"context_line":"interval is set as 60s. For every polling period, ceilometer-polling will"},{"line_number":27,"context_line":"invoke \"instance_get_all_by_host()\" method (and maybe need querying image and"},{"line_number":28,"context_line":"flavor also) to query the instances of this host from Nova API. That means"},{"line_number":29,"context_line":"the Nova API will receive 500 requests meanwhile every minutes. It would"},{"line_number":30,"context_line":"massively increase the workload of Nova API."},{"line_number":31,"context_line":""},{"line_number":32,"context_line":"In the metadata caching proposal[1], the instance info from Nova will be cached"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_4e0bc57d","line":29,"in_reply_to":"da20952f_e3b8c2c9","updated":"2015-09-14 09:51:57.000000000","message":"Done","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":7478,"name":"Nadya Shakhat","email":"nadmi4@gmail.com","username":"nprivalova"},"change_message_id":"0767a60e181c1814760bc5d93c3df1636d815169","unresolved":false,"context_lines":[{"line_number":30,"context_line":"massively increase the workload of Nova API."},{"line_number":31,"context_line":""},{"line_number":32,"context_line":"In the metadata caching proposal[1], the instance info from Nova will be cached"},{"line_number":33,"context_line":"in ceilometer-polling by using \u0027change-since\u0027 query filter of Nova server-list"},{"line_number":34,"context_line":"API. That will significantly decrease the amount of items of every Nova query."},{"line_number":35,"context_line":"But that cannot help to reduce the query frequency."},{"line_number":36,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_433f2e25","line":33,"updated":"2015-09-10 09:07:55.000000000","message":"That will significantly decrease \u003d\u003e That has significantly decreased\n\n(because it is already implemented)","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"5ead137180a3ead053cf4160029a79510099dd93","unresolved":false,"context_lines":[{"line_number":30,"context_line":"massively increase the workload of Nova API."},{"line_number":31,"context_line":""},{"line_number":32,"context_line":"In the metadata caching proposal[1], the instance info from Nova will be cached"},{"line_number":33,"context_line":"in ceilometer-polling by using \u0027change-since\u0027 query filter of Nova server-list"},{"line_number":34,"context_line":"API. That will significantly decrease the amount of items of every Nova query."},{"line_number":35,"context_line":"But that cannot help to reduce the query frequency."},{"line_number":36,"context_line":""}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_ae14e19e","line":33,"in_reply_to":"da20952f_433f2e25","updated":"2015-09-14 09:51:57.000000000","message":"Done","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":7478,"name":"Nadya Shakhat","email":"nadmi4@gmail.com","username":"nprivalova"},"change_message_id":"0767a60e181c1814760bc5d93c3df1636d815169","unresolved":false,"context_lines":[{"line_number":33,"context_line":"in ceilometer-polling by using \u0027change-since\u0027 query filter of Nova server-list"},{"line_number":34,"context_line":"API. That will significantly decrease the amount of items of every Nova query."},{"line_number":35,"context_line":"But that cannot help to reduce the query frequency."},{"line_number":36,"context_line":""},{"line_number":37,"context_line":"To dig the performance singularity of the instance related metering process,"},{"line_number":38,"context_line":"for the proposal[1] assuming we can deploy 100 instances on one host at most."},{"line_number":39,"context_line":"we can find it is not a very significant improvement reducing 100 instances to"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_03e5b6a6","line":36,"updated":"2015-09-10 09:07:55.000000000","message":"The following is not clear, tbh. Could you please try to make sentences shorter?","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"5ead137180a3ead053cf4160029a79510099dd93","unresolved":false,"context_lines":[{"line_number":33,"context_line":"in ceilometer-polling by using \u0027change-since\u0027 query filter of Nova server-list"},{"line_number":34,"context_line":"API. That will significantly decrease the amount of items of every Nova query."},{"line_number":35,"context_line":"But that cannot help to reduce the query frequency."},{"line_number":36,"context_line":""},{"line_number":37,"context_line":"To dig the performance singularity of the instance related metering process,"},{"line_number":38,"context_line":"for the proposal[1] assuming we can deploy 100 instances on one host at most."},{"line_number":39,"context_line":"we can find it is not a very significant improvement reducing 100 instances to"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_8e11ddb1","line":36,"in_reply_to":"da20952f_03e5b6a6","updated":"2015-09-14 09:51:57.000000000","message":"Done","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":7478,"name":"Nadya Shakhat","email":"nadmi4@gmail.com","username":"nprivalova"},"change_message_id":"0767a60e181c1814760bc5d93c3df1636d815169","unresolved":false,"context_lines":[{"line_number":45,"context_line":"Proposed change"},{"line_number":46,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"Currently, the *instance* samples is mainly generated by ceilometer-polling,"},{"line_number":49,"context_line":"the notification-agent also generate this samples but those will be deprecated."},{"line_number":50,"context_line":"The *instance* metric include the info of instance from in OpenStack layer."},{"line_number":51,"context_line":"The compute namespace ceilometer-polling is mainly used to collect the info"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_4302ce3f","line":48,"updated":"2015-09-10 09:07:55.000000000","message":"generate -\u003e generates; this -\u003e these","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"5ead137180a3ead053cf4160029a79510099dd93","unresolved":false,"context_lines":[{"line_number":45,"context_line":"Proposed change"},{"line_number":46,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":47,"context_line":""},{"line_number":48,"context_line":"Currently, the *instance* samples is mainly generated by ceilometer-polling,"},{"line_number":49,"context_line":"the notification-agent also generate this samples but those will be deprecated."},{"line_number":50,"context_line":"The *instance* metric include the info of instance from in OpenStack layer."},{"line_number":51,"context_line":"The compute namespace ceilometer-polling is mainly used to collect the info"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_ce1fd5b5","line":48,"in_reply_to":"da20952f_4302ce3f","updated":"2015-09-14 09:51:57.000000000","message":"Done","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":13997,"name":"SHIGEMATSU Mitsuhiro","email":"shigematsu.mitsuhiro@lab.ntt.co.jp","username":"pshige"},"change_message_id":"e2e65dae7eacbd797bc2c94b7bc6bd18e26486dc","unresolved":false,"context_lines":[{"line_number":55,"context_line":"main changes:"},{"line_number":56,"context_line":""},{"line_number":57,"context_line":"* Move the *instance* metric from compute namespace to central namespace of"},{"line_number":58,"context_line":"  ceilometer-polling. central polling will will use \"instance_get_all\" to query"},{"line_number":59,"context_line":"  all instance from Nova API and generate samples \"instance\". The"},{"line_number":60,"context_line":"  \"instance_get_all\" has also applied \"change-since\" parameter as query filter"},{"line_number":61,"context_line":"  to massively reduce the amount of every query and cache the instance info"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_4b17eaed","line":58,"updated":"2015-09-05 04:59:22.000000000","message":"s/will will/will/","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"5ead137180a3ead053cf4160029a79510099dd93","unresolved":false,"context_lines":[{"line_number":55,"context_line":"main changes:"},{"line_number":56,"context_line":""},{"line_number":57,"context_line":"* Move the *instance* metric from compute namespace to central namespace of"},{"line_number":58,"context_line":"  ceilometer-polling. central polling will will use \"instance_get_all\" to query"},{"line_number":59,"context_line":"  all instance from Nova API and generate samples \"instance\". The"},{"line_number":60,"context_line":"  \"instance_get_all\" has also applied \"change-since\" parameter as query filter"},{"line_number":61,"context_line":"  to massively reduce the amount of every query and cache the instance info"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_f6c7a295","line":58,"in_reply_to":"da20952f_4b17eaed","updated":"2015-09-14 09:51:57.000000000","message":"Done","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":13997,"name":"SHIGEMATSU Mitsuhiro","email":"shigematsu.mitsuhiro@lab.ntt.co.jp","username":"pshige"},"change_message_id":"e2e65dae7eacbd797bc2c94b7bc6bd18e26486dc","unresolved":false,"context_lines":[{"line_number":92,"context_line":"directly get from virt driver. these changes will avoid frequent Nova API"},{"line_number":93,"context_line":"requests and make the instance related samples more lightweight."},{"line_number":94,"context_line":""},{"line_number":95,"context_line":"It is cannot be ignored that the the polling solution for resource metadata"},{"line_number":96,"context_line":"won\u0027t be accurate and have delay of 1 polling period at most. But, all the"},{"line_number":97,"context_line":"polling solutions to collect data are not accurate and have delay. that depends"},{"line_number":98,"context_line":"on the acceptance of users. in production scenario, polling interval will be"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_6b142ee8","line":95,"updated":"2015-09-05 04:59:22.000000000","message":"s/the the/the/","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"5ead137180a3ead053cf4160029a79510099dd93","unresolved":false,"context_lines":[{"line_number":92,"context_line":"directly get from virt driver. these changes will avoid frequent Nova API"},{"line_number":93,"context_line":"requests and make the instance related samples more lightweight."},{"line_number":94,"context_line":""},{"line_number":95,"context_line":"It is cannot be ignored that the the polling solution for resource metadata"},{"line_number":96,"context_line":"won\u0027t be accurate and have delay of 1 polling period at most. But, all the"},{"line_number":97,"context_line":"polling solutions to collect data are not accurate and have delay. that depends"},{"line_number":98,"context_line":"on the acceptance of users. in production scenario, polling interval will be"}],"source_content_type":"text/x-rst","patch_set":4,"id":"da20952f_d6c13e81","line":95,"in_reply_to":"da20952f_6b142ee8","updated":"2015-09-14 09:51:57.000000000","message":"Done","commit_id":"5154595ed9f2c55ea0bfc0e9a91ed38589c710f2"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"e12f23f1294506ed893b46f76318e7fb8c841f5c","unresolved":false,"context_lines":[{"line_number":47,"context_line":"Proposed change"},{"line_number":48,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"Currently, the *instance* samples is mainly generated by ceilometer-polling,"},{"line_number":51,"context_line":"the notification-agent also generates these samples but those will be"},{"line_number":52,"context_line":"deprecated. The *instance* metric include the info of instance from in"},{"line_number":53,"context_line":"OpenStack layer. The compute namespace ceilometer-polling is mainly used to"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a2fa921_c20b8f7c","line":50,"updated":"2015-10-05 20:13:56.000000000","message":"they ideally will both be deprecated","commit_id":"793022b00c5964b94bc199d3306dc57c11577e57"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"43be3ac091c39d94c3417bae0101b872119b1a39","unresolved":false,"context_lines":[{"line_number":47,"context_line":"Proposed change"},{"line_number":48,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":49,"context_line":""},{"line_number":50,"context_line":"Currently, the *instance* samples is mainly generated by ceilometer-polling,"},{"line_number":51,"context_line":"the notification-agent also generates these samples but those will be"},{"line_number":52,"context_line":"deprecated. The *instance* metric include the info of instance from in"},{"line_number":53,"context_line":"OpenStack layer. The compute namespace ceilometer-polling is mainly used to"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a2fa921_dab5d3ec","line":50,"in_reply_to":"7a2fa921_c20b8f7c","updated":"2015-10-10 09:45:51.000000000","message":"yeah, this also block me, not sure it is a good idea to keep the *instance* meter in central namespace for instance metadata polling.\n\nwe have found that the current implementation of instance metrics polling also caused a heavy load of message bus because every sample almost include all the instance info, in this proposal, ceilometer-polling will generate lightweight samples. that hopefully this proposal can both reduce the load of Nova API and the load of message bus caused by ceilometer-polling.\n\nany suggestion ?","commit_id":"793022b00c5964b94bc199d3306dc57c11577e57"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"e12f23f1294506ed893b46f76318e7fb8c841f5c","unresolved":false,"context_lines":[{"line_number":87,"context_line":"  there is orphan instance has uuid, we can drop the samples before recording"},{"line_number":88,"context_line":"  to storage by comparing with *instance* samples recorded by central polling."},{"line_number":89,"context_line":""},{"line_number":90,"context_line":"The main idea of this proposal is to separate the resource metadata polling and"},{"line_number":91,"context_line":"metrics polling to central namespace and compute namespace. compute namespace"},{"line_number":92,"context_line":"ceilometer-polling will only focus on metering metrics about disk, cpu, net,"},{"line_number":93,"context_line":"mem, etc. the resource metadata collected by central polling will can be"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a2fa921_42aa1f1b","line":90,"updated":"2015-10-05 20:13:56.000000000","message":"what happens if we get metric before metadata? do we just store it without metadata?","commit_id":"793022b00c5964b94bc199d3306dc57c11577e57"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"43be3ac091c39d94c3417bae0101b872119b1a39","unresolved":false,"context_lines":[{"line_number":87,"context_line":"  there is orphan instance has uuid, we can drop the samples before recording"},{"line_number":88,"context_line":"  to storage by comparing with *instance* samples recorded by central polling."},{"line_number":89,"context_line":""},{"line_number":90,"context_line":"The main idea of this proposal is to separate the resource metadata polling and"},{"line_number":91,"context_line":"metrics polling to central namespace and compute namespace. compute namespace"},{"line_number":92,"context_line":"ceilometer-polling will only focus on metering metrics about disk, cpu, net,"},{"line_number":93,"context_line":"mem, etc. the resource metadata collected by central polling will can be"}],"source_content_type":"text/x-rst","patch_set":5,"id":"7a2fa921_bafc9772","line":90,"in_reply_to":"7a2fa921_42aa1f1b","updated":"2015-10-10 09:45:51.000000000","message":"I think the resource metadata is resource\u0027s attributes, not metrics\u0027s. so propose to store metrics and resource in separated database tables, and in separately recording processes. the stored resource and metrics will associated only when we need to query (or query statistics) filter by some resource attributes(user_id, project_id, metadata, etc.), and they associated with *resource_id*.","commit_id":"793022b00c5964b94bc199d3306dc57c11577e57"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"ddc02562188e358f981950b7923464f6b08a3cdb","unresolved":false,"context_lines":[{"line_number":37,"context_line":""},{"line_number":38,"context_line":"To find the performance singularity of the instance related metering process."},{"line_number":39,"context_line":"We assume that we can deploy 100 instances on one host at most. with"},{"line_number":40,"context_line":"proposal[1], that may reduce 100 instances to 20 instances returned in every"},{"line_number":41,"context_line":"Nova list call, it\u0027s not a significant improvement. But with this proposed"},{"line_number":42,"context_line":"change it may will reduce the times of Nova API request from 500 requests/min"},{"line_number":43,"context_line":"to 1 requests/min, it would be a significant improvement and that is the goal"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ba8a016a_019623f0","line":40,"range":{"start_line":40,"start_character":49,"end_line":40,"end_character":58},"updated":"2015-11-17 16:59:11.000000000","message":"did we actually test liberty or we just assuming?","commit_id":"e2203aa05d64658bac53d684303c8b2c11427ab2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"788efccac03787f193b6f3fafec16de5fc48fbee","unresolved":false,"context_lines":[{"line_number":37,"context_line":""},{"line_number":38,"context_line":"To find the performance singularity of the instance related metering process."},{"line_number":39,"context_line":"We assume that we can deploy 100 instances on one host at most. with"},{"line_number":40,"context_line":"proposal[1], that may reduce 100 instances to 20 instances returned in every"},{"line_number":41,"context_line":"Nova list call, it\u0027s not a significant improvement. But with this proposed"},{"line_number":42,"context_line":"change it may will reduce the times of Nova API request from 500 requests/min"},{"line_number":43,"context_line":"to 1 requests/min, it would be a significant improvement and that is the goal"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ba8a016a_91047dc0","line":40,"in_reply_to":"ba8a016a_019623f0","updated":"2015-11-18 09:10:21.000000000","message":"sorry, just assuming, but I think this situation is related the state of a deployment, maybe the updated instance in a polling cycle is much less, even the returned instance is a empty list, it will be a improvement, but it will also cause a plenty of Nova API calls meanwhile and the amount related to the scale of deployment.\n\nadditionally, the samples without instance resource metadata will reduce the size of payload of notification from ceilometer-polling, I think it is also a improvement to reduce the load of message bus.\n\nactually, i have radical idea that all the metric data should be very simple, include the value (volume), timestamp, unit, etc. personally think these attributes is enough for metrics data and the mean of metric data, the resource info should be separated, and they are related with *resource_id*, this is more like the indexer and metrics storage in Gnocchi. but I am not sure this idea make any sense to others :) I remember the metric definition in Monasca is very simple.","commit_id":"e2203aa05d64658bac53d684303c8b2c11427ab2"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"ddc02562188e358f981950b7923464f6b08a3cdb","unresolved":false,"context_lines":[{"line_number":57,"context_line":"According this, this proposal try to solve the problem described above by two"},{"line_number":58,"context_line":"main changes:"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"* As we have decided to deprecate the \"volume\u003d1\" meter, the *instance* meter"},{"line_number":61,"context_line":"  will be only used to collect resource(instance) metadata. We propose to add"},{"line_number":62,"context_line":"  a cache mechanism to notification-agent used to cache resource metadata."},{"line_number":63,"context_line":"  And then move the *instance* meter from compute namespace to central"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ba8a016a_815f7340","line":60,"range":{"start_line":60,"start_character":24,"end_line":60,"end_character":33},"updated":"2015-11-17 16:59:11.000000000","message":"at summit, we decided to keep the volume\u003d1 samples but not stored them","commit_id":"e2203aa05d64658bac53d684303c8b2c11427ab2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"788efccac03787f193b6f3fafec16de5fc48fbee","unresolved":false,"context_lines":[{"line_number":57,"context_line":"According this, this proposal try to solve the problem described above by two"},{"line_number":58,"context_line":"main changes:"},{"line_number":59,"context_line":""},{"line_number":60,"context_line":"* As we have decided to deprecate the \"volume\u003d1\" meter, the *instance* meter"},{"line_number":61,"context_line":"  will be only used to collect resource(instance) metadata. We propose to add"},{"line_number":62,"context_line":"  a cache mechanism to notification-agent used to cache resource metadata."},{"line_number":63,"context_line":"  And then move the *instance* meter from compute namespace to central"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ba8a016a_512db544","line":60,"in_reply_to":"ba8a016a_815f7340","updated":"2015-11-18 09:10:21.000000000","message":"yeah, I will modify this description.","commit_id":"e2203aa05d64658bac53d684303c8b2c11427ab2"},{"author":{"_account_id":6794,"name":"Simon Pasquier","email":"pasquier.simon@gmail.com","username":"simon-pasquier"},"change_message_id":"65dc720073bda6ea75c2654b8f31fdb62181f9a1","unresolved":false,"context_lines":[{"line_number":93,"context_line":"  (called orphan instances), these need to be dropped. Firstly, we can check"},{"line_number":94,"context_line":"  the vm info in virt layer to filter out the orphan instances, see[2]."},{"line_number":95,"context_line":"  Secondly, when instance metrics are send from ceilometer-polling to"},{"line_number":96,"context_line":"  notification-agent, the resource_id of the metric dose not exist in resource"},{"line_number":97,"context_line":"  cache, we should drop it."},{"line_number":98,"context_line":""},{"line_number":99,"context_line":"The main idea of this proposal is to separate the resource metadata polling and"}],"source_content_type":"text/x-rst","patch_set":6,"id":"da85f559_ba2c1103","line":96,"range":{"start_line":96,"start_character":52,"end_line":96,"end_character":56},"updated":"2015-11-10 11:05:14.000000000","message":"does","commit_id":"e2203aa05d64658bac53d684303c8b2c11427ab2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"788efccac03787f193b6f3fafec16de5fc48fbee","unresolved":false,"context_lines":[{"line_number":93,"context_line":"  (called orphan instances), these need to be dropped. Firstly, we can check"},{"line_number":94,"context_line":"  the vm info in virt layer to filter out the orphan instances, see[2]."},{"line_number":95,"context_line":"  Secondly, when instance metrics are send from ceilometer-polling to"},{"line_number":96,"context_line":"  notification-agent, the resource_id of the metric dose not exist in resource"},{"line_number":97,"context_line":"  cache, we should drop it."},{"line_number":98,"context_line":""},{"line_number":99,"context_line":"The main idea of this proposal is to separate the resource metadata polling and"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ba8a016a_7103190e","line":96,"in_reply_to":"da85f559_ba2c1103","updated":"2015-11-18 09:10:21.000000000","message":"Done","commit_id":"e2203aa05d64658bac53d684303c8b2c11427ab2"},{"author":{"_account_id":4491,"name":"Lianhao Lu","email":"llh_misc@outlook.com","username":"lianhao-lu"},"change_message_id":"3e3f6648a39f3350c70dabf561e8e89d740b03c0","unresolved":false,"context_lines":[{"line_number":102,"context_line":"mem, etc. the resource metadata collected by central polling will can be"},{"line_number":103,"context_line":"associated with samples by unique \"resource_id\", the resource_id can be"},{"line_number":104,"context_line":"directly get from virt driver. these changes will avoid frequent Nova API"},{"line_number":105,"context_line":"requests and make the instance related samples more lightweight."},{"line_number":106,"context_line":""},{"line_number":107,"context_line":"It is cannot be ignored that the polling solution for resource metadata"},{"line_number":108,"context_line":"won\u0027t be accurate and have delay of 1 polling period at most. But, all the"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ba8a016a_19f44185","line":105,"updated":"2015-11-23 22:38:21.000000000","message":"Can we move all the things happens on central agent to compute agent and combine the metadata and metering metrics on compute agent? I think it\u0027s much more simple.\n\nb.t.w. Can we detect the metadata change from notificaiton?","commit_id":"e2203aa05d64658bac53d684303c8b2c11427ab2"},{"author":{"_account_id":4491,"name":"Lianhao Lu","email":"llh_misc@outlook.com","username":"lianhao-lu"},"change_message_id":"7e817c31b7477cf5e57e7b86e20e87feb959fd05","unresolved":false,"context_lines":[{"line_number":102,"context_line":"mem, etc. the resource metadata collected by central polling will can be"},{"line_number":103,"context_line":"associated with samples by unique \"resource_id\", the resource_id can be"},{"line_number":104,"context_line":"directly get from virt driver. these changes will avoid frequent Nova API"},{"line_number":105,"context_line":"requests and make the instance related samples more lightweight."},{"line_number":106,"context_line":""},{"line_number":107,"context_line":"It is cannot be ignored that the polling solution for resource metadata"},{"line_number":108,"context_line":"won\u0027t be accurate and have delay of 1 polling period at most. But, all the"}],"source_content_type":"text/x-rst","patch_set":6,"id":"9a8ffd7b_c465d2e1","line":105,"in_reply_to":"ba8a016a_00d036f5","updated":"2015-11-24 15:28:28.000000000","message":"Did you mention that you could get instance data by looking through hypervisor instead of nova api? How about just query the metadata for those instances which is active on the compute node?","commit_id":"e2203aa05d64658bac53d684303c8b2c11427ab2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"d91e379739dca6331be4fccbdcfb998d0f8c4615","unresolved":false,"context_lines":[{"line_number":102,"context_line":"mem, etc. the resource metadata collected by central polling will can be"},{"line_number":103,"context_line":"associated with samples by unique \"resource_id\", the resource_id can be"},{"line_number":104,"context_line":"directly get from virt driver. these changes will avoid frequent Nova API"},{"line_number":105,"context_line":"requests and make the instance related samples more lightweight."},{"line_number":106,"context_line":""},{"line_number":107,"context_line":"It is cannot be ignored that the polling solution for resource metadata"},{"line_number":108,"context_line":"won\u0027t be accurate and have delay of 1 polling period at most. But, all the"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ba8a016a_00d036f5","line":105,"in_reply_to":"ba8a016a_19f44185","updated":"2015-11-24 07:56:26.000000000","message":"the compute agents(ceilometer-polling) run on every compute node, if we combine metadata metering data and resource data on compute agent, we need a way to provide resource(instance) info to every compute node.\n\nthe way of metadata change from notification, I have explained in \"Alternatives\" :)","commit_id":"e2203aa05d64658bac53d684303c8b2c11427ab2"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"ddc02562188e358f981950b7923464f6b08a3cdb","unresolved":false,"context_lines":[{"line_number":213,"context_line":"Dependencies"},{"line_number":214,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":215,"context_line":""},{"line_number":216,"context_line":"None."},{"line_number":217,"context_line":""},{"line_number":218,"context_line":"Testing"},{"line_number":219,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ba8a016a_01ef0388","line":216,"range":{"start_line":216,"start_character":4,"end_line":216,"end_character":5},"updated":"2015-11-17 16:59:11.000000000","message":"will this not require a global cache? if there is multiple notificaiton agents, there is no way to guarantee your central agent polled metadata will end up on all notification agents.\n\ngiven this requirement for global cache, can\u0027t we just connect the pollsters together and ignore the notification agent completely?","commit_id":"e2203aa05d64658bac53d684303c8b2c11427ab2"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"788efccac03787f193b6f3fafec16de5fc48fbee","unresolved":false,"context_lines":[{"line_number":213,"context_line":"Dependencies"},{"line_number":214,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":215,"context_line":""},{"line_number":216,"context_line":"None."},{"line_number":217,"context_line":""},{"line_number":218,"context_line":"Testing"},{"line_number":219,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":6,"id":"ba8a016a_4cc8be22","line":216,"in_reply_to":"ba8a016a_01ef0388","updated":"2015-11-18 09:10:21.000000000","message":"yes, thanks for reminding me, we should consider the multiple notification agents scenario.\n\nIIUC, the global cache you said will be used by all polling agents directly, it is looks like to centralize the resource metadata cache than the distributed cache implemented by proposal[1], right? not sure this will cause another performance problem, and should we consider the workload of message bus caused by ceilometer-polling I replied you above ?","commit_id":"e2203aa05d64658bac53d684303c8b2c11427ab2"},{"author":{"_account_id":1669,"name":"Julien Danjou","display_name":"jd","email":"julien@danjou.info","username":"jdanjou"},"change_message_id":"31ce4d63d1e01b3ee77717966615018ea8378f1d","unresolved":false,"context_lines":[{"line_number":27,"context_line":"interval is set to 60s. For every polling period, ceilometer-polling will"},{"line_number":28,"context_line":"invoke \"instance_get_all_by_host()\" method (and maybe need querying image and"},{"line_number":29,"context_line":"flavor also) to query the instances of this host from Nova API. That means"},{"line_number":30,"context_line":"the Nova API will receive 500 requests meanwhile every minutes. It would"},{"line_number":31,"context_line":"massively increase the workload on Nova API."},{"line_number":32,"context_line":""},{"line_number":33,"context_line":"Additionally, the metric data about instances also take with the instance"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da6ed579_8318881f","line":30,"updated":"2016-01-14 15:17:05.000000000","message":"You realize that this is actually a problem with Nova? A simple select to retrieve all of this data is \u003c1s in any correct RDBMS properly used.\n\n500 rows to select is *nothing*. The problem is really with Nova doing a poor job here. But ok :)","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"beafc1ba82f78b0fc4c8f59c17319e5e683a71f8","unresolved":false,"context_lines":[{"line_number":27,"context_line":"interval is set to 60s. For every polling period, ceilometer-polling will"},{"line_number":28,"context_line":"invoke \"instance_get_all_by_host()\" method (and maybe need querying image and"},{"line_number":29,"context_line":"flavor also) to query the instances of this host from Nova API. That means"},{"line_number":30,"context_line":"the Nova API will receive 500 requests meanwhile every minutes. It would"},{"line_number":31,"context_line":"massively increase the workload on Nova API."},{"line_number":32,"context_line":""},{"line_number":33,"context_line":"Additionally, the metric data about instances also take with the instance"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da6ed579_5aa1d8f6","line":30,"updated":"2016-01-15 13:57:53.000000000","message":"jd, I think you\u0027ve misread this. The issue is that 500 requests simultaneously land on the nova-api every 60s. Even with the \u0027changes-since\u0027 change that Jason did that\u0027s still a hit.\n\nOf course I\u0027m not disagreeing with you that Nova\u0027s data storage is weak sauce, but it\u0027s not 500 rows we\u0027re talking about here. It\u0027s 500 different requests of the same table, each returning a different set of rows.\n\nAnd of course, Nova should be able to handle this kind of request (at much higher rates) but it doesn\u0027t. So, in the meantime...","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"8b6fa1d8bff4cce61a9818649bb98f090035e20f","unresolved":false,"context_lines":[{"line_number":27,"context_line":"interval is set to 60s. For every polling period, ceilometer-polling will"},{"line_number":28,"context_line":"invoke \"instance_get_all_by_host()\" method (and maybe need querying image and"},{"line_number":29,"context_line":"flavor also) to query the instances of this host from Nova API. That means"},{"line_number":30,"context_line":"the Nova API will receive 500 requests meanwhile every minutes. It would"},{"line_number":31,"context_line":"massively increase the workload on Nova API."},{"line_number":32,"context_line":""},{"line_number":33,"context_line":"Additionally, the metric data about instances also take with the instance"}],"source_content_type":"text/x-rst","patch_set":7,"id":"ba63e191_0608722b","line":30,"in_reply_to":"da6ed579_5aa1d8f6","updated":"2016-01-21 11:50:43.000000000","message":"thanks for explanation, 500 is just a example :), that may be larger scale. and it seems also need to request images (glance API) and instance flavors.","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":1669,"name":"Julien Danjou","display_name":"jd","email":"julien@danjou.info","username":"jdanjou"},"change_message_id":"31ce4d63d1e01b3ee77717966615018ea8378f1d","unresolved":false,"context_lines":[{"line_number":35,"context_line":"increase the workload of message queue when publishing it and the IO load when"},{"line_number":36,"context_line":"recording it to storage."},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"In the metadata caching proposal[1], the instance info from Nova will be cached"},{"line_number":39,"context_line":"in ceilometer-polling by using \u0027change-since\u0027 query filter of Nova server-list"},{"line_number":40,"context_line":"API. That has significantly decreased the amount of items of every Nova query."},{"line_number":41,"context_line":"But that cannot help to reduce the query frequency."}],"source_content_type":"text/x-rst","patch_set":7,"id":"da6ed579_43913046","line":38,"updated":"2016-01-14 15:17:05.000000000","message":"this has been implemented AFAIK, so it\u0027s not \"will be\" but \"is\", right?","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"8b6fa1d8bff4cce61a9818649bb98f090035e20f","unresolved":false,"context_lines":[{"line_number":35,"context_line":"increase the workload of message queue when publishing it and the IO load when"},{"line_number":36,"context_line":"recording it to storage."},{"line_number":37,"context_line":""},{"line_number":38,"context_line":"In the metadata caching proposal[1], the instance info from Nova will be cached"},{"line_number":39,"context_line":"in ceilometer-polling by using \u0027change-since\u0027 query filter of Nova server-list"},{"line_number":40,"context_line":"API. That has significantly decreased the amount of items of every Nova query."},{"line_number":41,"context_line":"But that cannot help to reduce the query frequency."}],"source_content_type":"text/x-rst","patch_set":7,"id":"ba63e191_c1000404","line":38,"in_reply_to":"da6ed579_43913046","updated":"2016-01-21 11:50:43.000000000","message":"Done","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"beafc1ba82f78b0fc4c8f59c17319e5e683a71f8","unresolved":false,"context_lines":[{"line_number":40,"context_line":"API. That has significantly decreased the amount of items of every Nova query."},{"line_number":41,"context_line":"But that cannot help to reduce the query frequency."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"To find the performance singularity of the instance related metering process."},{"line_number":44,"context_line":"We assume that we can deploy 100 instances on one host at most. with"},{"line_number":45,"context_line":"proposal[1], that may reduce 100 instances to 20 instances returned in every"},{"line_number":46,"context_line":"Nova list call, it\u0027s not a significant improvement. But with this proposed"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da6ed579_faa3ecfd","line":43,"updated":"2016-01-15 13:57:53.000000000","message":"This sentence is incomplete. Or at least I don\u0027t understand it.","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"8b6fa1d8bff4cce61a9818649bb98f090035e20f","unresolved":false,"context_lines":[{"line_number":40,"context_line":"API. That has significantly decreased the amount of items of every Nova query."},{"line_number":41,"context_line":"But that cannot help to reduce the query frequency."},{"line_number":42,"context_line":""},{"line_number":43,"context_line":"To find the performance singularity of the instance related metering process."},{"line_number":44,"context_line":"We assume that we can deploy 100 instances on one host at most. with"},{"line_number":45,"context_line":"proposal[1], that may reduce 100 instances to 20 instances returned in every"},{"line_number":46,"context_line":"Nova list call, it\u0027s not a significant improvement. But with this proposed"}],"source_content_type":"text/x-rst","patch_set":7,"id":"ba63e191_a17150e2","line":43,"in_reply_to":"da6ed579_faa3ecfd","updated":"2016-01-21 11:50:43.000000000","message":"I will modify it :)","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":7478,"name":"Nadya Shakhat","email":"nadmi4@gmail.com","username":"nprivalova"},"change_message_id":"9e98d35b0d5a86b2502e7afb948a159c3c6213b5","unresolved":false,"context_lines":[{"line_number":75,"context_line":""},{"line_number":76,"context_line":"  NOTE(TBD): the resource metadata cache mechanism should consider multiple"},{"line_number":77,"context_line":"  notification-agent instances in a deployment, that means the cache should be"},{"line_number":78,"context_line":"  a global cache. The details of this cache mechanism will be described in"},{"line_number":79,"context_line":"  another spec, spec[3]  or spec[4] ?"},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"* The ceilometer-polling running on compute nodes with \"compute\" namespace"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3a7e1126_5de74ee4","line":78,"range":{"start_line":78,"start_character":0,"end_line":78,"end_character":74},"updated":"2015-12-17 13:13:32.000000000","message":"From my point of view, I think it\u0027s possible using Redis and sorted sets. Lem me explain.\n\nIn HBase  driver we had the same problem about metadata. When user does resource_show, Ceilometer should return the current metadata for this resource. But actually, samples may arrive out-of-order. That\u0027s why when a new sample arrives, the algorithm should be the following: 1. get corresponding resource from db 2. check that stored metadata for this resource is actual 3. if it is, don\u0027t update the resource. if the metadata of the new sample is newer, update the resource in db\n\nAnd all these steps should be done in one transaction. But there is no transactions in HBase. That\u0027s why we used versioning. in HBase it\u0027s possible to set timestamp of value written to db manually. Using this approach, it\u0027s possible just write to key \u003cresource_id\u003e metadata with timestamp. All metadata will be automatically sorted by timestamp. When we do show_resource, HBase returns metadata from the top.\n\nSpeaking about metadata caching, we need to be able not only get the top metadata, but the closest to the current sample. Because it\u0027s not correct to associate metadata arrived on 1st of January with the sample arrived on 31 of December (I\u0027m considering the corner case when we process X messages from MQ after several days without notification agents)\n\nBased on my researches, Redis sorted sets may be used for this scenario","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":1669,"name":"Julien Danjou","display_name":"jd","email":"julien@danjou.info","username":"jdanjou"},"change_message_id":"31ce4d63d1e01b3ee77717966615018ea8378f1d","unresolved":false,"context_lines":[{"line_number":75,"context_line":""},{"line_number":76,"context_line":"  NOTE(TBD): the resource metadata cache mechanism should consider multiple"},{"line_number":77,"context_line":"  notification-agent instances in a deployment, that means the cache should be"},{"line_number":78,"context_line":"  a global cache. The details of this cache mechanism will be described in"},{"line_number":79,"context_line":"  another spec, spec[3]  or spec[4] ?"},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"* The ceilometer-polling running on compute nodes with \"compute\" namespace"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da6ed579_a31b4cc9","line":78,"range":{"start_line":78,"start_character":0,"end_line":78,"end_character":74},"in_reply_to":"3a7e1126_035ff90e","updated":"2016-01-14 15:17:05.000000000","message":"There is an oslo module to do cache, maybe we should use it?","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"b848981fde61ce9879174dedb843c1ec67e8a702","unresolved":false,"context_lines":[{"line_number":75,"context_line":""},{"line_number":76,"context_line":"  NOTE(TBD): the resource metadata cache mechanism should consider multiple"},{"line_number":77,"context_line":"  notification-agent instances in a deployment, that means the cache should be"},{"line_number":78,"context_line":"  a global cache. The details of this cache mechanism will be described in"},{"line_number":79,"context_line":"  another spec, spec[3]  or spec[4] ?"},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"* The ceilometer-polling running on compute nodes with \"compute\" namespace"}],"source_content_type":"text/x-rst","patch_set":7,"id":"3a7e1126_035ff90e","line":78,"range":{"start_line":78,"start_character":0,"end_line":78,"end_character":74},"in_reply_to":"3a7e1126_5de74ee4","updated":"2015-12-21 03:31:41.000000000","message":"Hi Nadya,\n\nFirstly, very sorry for missing your ping in IRC :(\n\nThanks for you researches, for the cache mechanism, yeah, in Gnocchi and impl_sqlalchemy, we have made the resource and metric data stored separated, the resource metadata cache will be update by polling (with \"change-since\") in central namespace, so if we use this proposal + resource metadata cache notification-agent, the samples will be assembled by metric data from virt layer and metadata from cache, and the samples object will be same as current.\n\nFor the cache implementation, I am not very familiar with Redis, thanks for your explanation, may we can consider to use oslo_cache as gord and cdent suggested ?","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"beafc1ba82f78b0fc4c8f59c17319e5e683a71f8","unresolved":false,"context_lines":[{"line_number":104,"context_line":"  the vm info in virt layer to filter out the orphan instances, see[2]."},{"line_number":105,"context_line":"  Secondly, when instance metrics are send from ceilometer-polling to"},{"line_number":106,"context_line":"  notification-agent, the resource_id of the metric does not exist in resource"},{"line_number":107,"context_line":"  cache, we should drop it."},{"line_number":108,"context_line":""},{"line_number":109,"context_line":"The main idea of this proposal is to separate the resource metadata polling and"},{"line_number":110,"context_line":"metrics polling to central namespace and compute namespace. compute namespace"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da6ed579_1aa7e00b","line":107,"updated":"2016-01-15 13:57:53.000000000","message":"Yes","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"beafc1ba82f78b0fc4c8f59c17319e5e683a71f8","unresolved":false,"context_lines":[{"line_number":116,"context_line":"change will reduce the workload of message bus caused metrics publishing and"},{"line_number":117,"context_line":"reduce the IO load caused by metrics data recording."},{"line_number":118,"context_line":""},{"line_number":119,"context_line":"It is cannot be ignored that the polling solution for resource metadata"},{"line_number":120,"context_line":"won\u0027t be accurate and have delay of 1 polling period at most. But, all the"},{"line_number":121,"context_line":"polling solutions to collect data are not accurate and have delay. that depends"},{"line_number":122,"context_line":"on the acceptance of users. in production scenario, polling interval will be"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da6ed579_bab2544a","line":119,"updated":"2016-01-15 13:57:53.000000000","message":"s/is//","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"8b6fa1d8bff4cce61a9818649bb98f090035e20f","unresolved":false,"context_lines":[{"line_number":116,"context_line":"change will reduce the workload of message bus caused metrics publishing and"},{"line_number":117,"context_line":"reduce the IO load caused by metrics data recording."},{"line_number":118,"context_line":""},{"line_number":119,"context_line":"It is cannot be ignored that the polling solution for resource metadata"},{"line_number":120,"context_line":"won\u0027t be accurate and have delay of 1 polling period at most. But, all the"},{"line_number":121,"context_line":"polling solutions to collect data are not accurate and have delay. that depends"},{"line_number":122,"context_line":"on the acceptance of users. in production scenario, polling interval will be"}],"source_content_type":"text/x-rst","patch_set":7,"id":"ba63e191_417974c7","line":119,"in_reply_to":"da6ed579_bab2544a","updated":"2016-01-21 11:50:43.000000000","message":"Done","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"beafc1ba82f78b0fc4c8f59c17319e5e683a71f8","unresolved":false,"context_lines":[{"line_number":128,"context_line":""},{"line_number":129,"context_line":"Another suggested solution for this goal is using instance events by"},{"line_number":130,"context_line":"notifications instead of instance metrics by polling to collect instance"},{"line_number":131,"context_line":"resource metadata."},{"line_number":132,"context_line":""},{"line_number":133,"context_line":"Using events to update resource metadata in ceilometer or Gnocchi will be more"},{"line_number":134,"context_line":"efficient because the resource metadata may only need to be update when"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da6ed579_daadc8e9","line":131,"updated":"2016-01-15 13:57:53.000000000","message":"I was just going to ask about this...","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"beafc1ba82f78b0fc4c8f59c17319e5e683a71f8","unresolved":false,"context_lines":[{"line_number":136,"context_line":"approach:"},{"line_number":137,"context_line":""},{"line_number":138,"context_line":"* The resource metadata updating will depend on events collection, what if some"},{"line_number":139,"context_line":"  events didn\u0027t be collected ?"},{"line_number":140,"context_line":"* We need to ensure all the resource update actions in other projects of"},{"line_number":141,"context_line":"  OpenStack have corresponding events published, that need other resources"},{"line_number":142,"context_line":"  providers to ensure."}],"source_content_type":"text/x-rst","patch_set":7,"id":"da6ed579_7ab85c2a","line":139,"updated":"2016-01-15 13:57:53.000000000","message":"Maybe the notification agent should always subscribe to instance events, even if event storage is turned off?","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"8b6fa1d8bff4cce61a9818649bb98f090035e20f","unresolved":false,"context_lines":[{"line_number":136,"context_line":"approach:"},{"line_number":137,"context_line":""},{"line_number":138,"context_line":"* The resource metadata updating will depend on events collection, what if some"},{"line_number":139,"context_line":"  events didn\u0027t be collected ?"},{"line_number":140,"context_line":"* We need to ensure all the resource update actions in other projects of"},{"line_number":141,"context_line":"  OpenStack have corresponding events published, that need other resources"},{"line_number":142,"context_line":"  providers to ensure."}],"source_content_type":"text/x-rst","patch_set":7,"id":"9a68dd71_c9d85f56","line":139,"in_reply_to":"da6ed579_7ab85c2a","updated":"2016-01-21 11:50:43.000000000","message":"all the instance related metering will depend on the Nova events notifications.","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"beafc1ba82f78b0fc4c8f59c17319e5e683a71f8","unresolved":false,"context_lines":[{"line_number":139,"context_line":"  events didn\u0027t be collected ?"},{"line_number":140,"context_line":"* We need to ensure all the resource update actions in other projects of"},{"line_number":141,"context_line":"  OpenStack have corresponding events published, that need other resources"},{"line_number":142,"context_line":"  providers to ensure."},{"line_number":143,"context_line":"* Does all the events include enough info for ceilometer? we may need the"},{"line_number":144,"context_line":"  resource metadata do some filter query. e.g. use query filtered by instance"},{"line_number":145,"context_line":"  metadata do auto-scaling."}],"source_content_type":"text/x-rst","patch_set":7,"id":"da6ed579_9ab3d04c","line":142,"updated":"2016-01-15 13:57:53.000000000","message":"I\u0027m unable to parse this paragraph.","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"8b6fa1d8bff4cce61a9818649bb98f090035e20f","unresolved":false,"context_lines":[{"line_number":139,"context_line":"  events didn\u0027t be collected ?"},{"line_number":140,"context_line":"* We need to ensure all the resource update actions in other projects of"},{"line_number":141,"context_line":"  OpenStack have corresponding events published, that need other resources"},{"line_number":142,"context_line":"  providers to ensure."},{"line_number":143,"context_line":"* Does all the events include enough info for ceilometer? we may need the"},{"line_number":144,"context_line":"  resource metadata do some filter query. e.g. use query filtered by instance"},{"line_number":145,"context_line":"  metadata do auto-scaling."}],"source_content_type":"text/x-rst","patch_set":7,"id":"9a68dd71_7e3b27ec","line":142,"in_reply_to":"da6ed579_9ab3d04c","updated":"2016-01-21 11:50:43.000000000","message":"I meant if we use events to trigger resource metadata updating, we must sure all the updates of resources should emit a events received by Ceilometer.","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"beafc1ba82f78b0fc4c8f59c17319e5e683a71f8","unresolved":false,"context_lines":[{"line_number":140,"context_line":"* We need to ensure all the resource update actions in other projects of"},{"line_number":141,"context_line":"  OpenStack have corresponding events published, that need other resources"},{"line_number":142,"context_line":"  providers to ensure."},{"line_number":143,"context_line":"* Does all the events include enough info for ceilometer? we may need the"},{"line_number":144,"context_line":"  resource metadata do some filter query. e.g. use query filtered by instance"},{"line_number":145,"context_line":"  metadata do auto-scaling."},{"line_number":146,"context_line":""}],"source_content_type":"text/x-rst","patch_set":7,"id":"da6ed579_3a850459","line":143,"updated":"2016-01-15 13:57:53.000000000","message":"s/Does/Do/","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"8b6fa1d8bff4cce61a9818649bb98f090035e20f","unresolved":false,"context_lines":[{"line_number":140,"context_line":"* We need to ensure all the resource update actions in other projects of"},{"line_number":141,"context_line":"  OpenStack have corresponding events published, that need other resources"},{"line_number":142,"context_line":"  providers to ensure."},{"line_number":143,"context_line":"* Does all the events include enough info for ceilometer? we may need the"},{"line_number":144,"context_line":"  resource metadata do some filter query. e.g. use query filtered by instance"},{"line_number":145,"context_line":"  metadata do auto-scaling."},{"line_number":146,"context_line":""}],"source_content_type":"text/x-rst","patch_set":7,"id":"ba63e191_c14b2486","line":143,"in_reply_to":"da6ed579_3a850459","updated":"2016-01-21 11:50:43.000000000","message":"Done","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":11564,"name":"Chris Dent","email":"cdent@anticdent.org","username":"chdent"},"change_message_id":"beafc1ba82f78b0fc4c8f59c17319e5e683a71f8","unresolved":false,"context_lines":[{"line_number":226,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":227,"context_line":""},{"line_number":228,"context_line":"TBD: this proposal will depend on the global cache in notification-agent"},{"line_number":229,"context_line":"proposal, proposal[3] or proposal[4] ?"},{"line_number":230,"context_line":""},{"line_number":231,"context_line":""},{"line_number":232,"context_line":"Testing"}],"source_content_type":"text/x-rst","patch_set":7,"id":"da6ed579_5a88f870","line":229,"updated":"2016-01-15 13:57:53.000000000","message":"I think it would be worth implementing this with cache local to the notification-agent, first, if that\u0027s possible. Then pursue global cache if it seems to be working. That would still shrink the number of hits on nova-api by a ton.","commit_id":"be9b19ff699996013969665305bf1a5b692344f1"},{"author":{"_account_id":7729,"name":"Ilya Tyaptin","email":"ityaptin@mirantis.com","username":"ityaptin"},"change_message_id":"b9904e61d6af03e46a3d4fb0785b7168a02469a3","unresolved":false,"context_lines":[{"line_number":48,"context_line":""},{"line_number":49,"context_line":"This proposal try to solve the problem described above with changes:"},{"line_number":50,"context_line":""},{"line_number":51,"context_line":"* The ceilometer-polling agent with \"compute\" namespace runs on every compute"},{"line_number":52,"context_line":"  compute nodes. Currently, the agent need to get the instances list of this"},{"line_number":53,"context_line":"  host by calling \"instance_get_all_by_host\" method of novaclient, then traverse"},{"line_number":54,"context_line":"  this instance list to get metrics of instance(e.g. cpu, disk.read.bytes,"},{"line_number":55,"context_line":"  network.incoming.bytes) by inspector, which will invoke the interfaces of"}],"source_content_type":"text/x-rst","patch_set":8,"id":"9a68dd71_ccc5b68e","line":52,"range":{"start_line":51,"start_character":69,"end_line":52,"end_character":9},"updated":"2016-01-21 15:55:37.000000000","message":"\"compute\" repeated.","commit_id":"3cb72818e6e53ee310f889c47c311ea2fa62fceb"},{"author":{"_account_id":7478,"name":"Nadya Shakhat","email":"nadmi4@gmail.com","username":"nprivalova"},"change_message_id":"3209bd1b135698616a588201d8f2ccb281111d1d","unresolved":false,"context_lines":[{"line_number":61,"context_line":"  resource metadata(instance info from Nova db). ceilometer-polling will use"},{"line_number":62,"context_line":"  \"instance_get_all\" to query all the instance from Nova, and then cache them"},{"line_number":63,"context_line":"  to the global cache. Also, ceilometer-polling will record the resource"},{"line_number":64,"context_line":"  caching time to the cache system, and add an option named"},{"line_number":65,"context_line":"  \"interval_to_update_resource\" to indicate the max interval of resource cache"},{"line_number":66,"context_line":"  updating. If the time from last updating is larger than this option value,"},{"line_number":67,"context_line":"  the resource cache need to be updated again."}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a68dd71_b0efe71c","line":64,"updated":"2016-01-25 13:38:52.000000000","message":"Will we update the cache using notifications as well? Not only polling once in  interval_to_update_resource, but update during notification processing?\n...\nI\u0027ve read your spec once more and looks like I understood that my suggestion may be not good. Even if we update the cache using events, we still need to use change_since\u003dlast_time_we_updated_the_cache_using_polling because we may lost (assume that we may) information from other resources. But anyway, if we do use notification events, samples will be more up_to_date. because the cache will contain the most recent info. So, yes, we will do the same amount of requests to Nova, but the data will be more accurate.","commit_id":"43298c480a33e4945baeb6b2bfc4d92d425193eb"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"994fd2759819fe07db00c638e5c65129128aacb2","unresolved":false,"context_lines":[{"line_number":61,"context_line":"  resource metadata(instance info from Nova db). ceilometer-polling will use"},{"line_number":62,"context_line":"  \"instance_get_all\" to query all the instance from Nova, and then cache them"},{"line_number":63,"context_line":"  to the global cache. Also, ceilometer-polling will record the resource"},{"line_number":64,"context_line":"  caching time to the cache system, and add an option named"},{"line_number":65,"context_line":"  \"interval_to_update_resource\" to indicate the max interval of resource cache"},{"line_number":66,"context_line":"  updating. If the time from last updating is larger than this option value,"},{"line_number":67,"context_line":"  the resource cache need to be updated again."}],"source_content_type":"text/x-rst","patch_set":9,"id":"7a5de9d1_f6ac21ef","line":64,"in_reply_to":"7a5de9d1_7337eeb0","updated":"2016-01-28 02:57:46.000000000","message":"maybe the resource updating based on notification be another optional functionality if we want more real-time  resource metadata, but I afraid that will also increase the load of notification-agent especially some batching operations in Nova occurred.","commit_id":"43298c480a33e4945baeb6b2bfc4d92d425193eb"},{"author":{"_account_id":4491,"name":"Lianhao Lu","email":"llh_misc@outlook.com","username":"lianhao-lu"},"change_message_id":"8821fb67e3054399b79565132b3abf24972e9ee9","unresolved":false,"context_lines":[{"line_number":61,"context_line":"  resource metadata(instance info from Nova db). ceilometer-polling will use"},{"line_number":62,"context_line":"  \"instance_get_all\" to query all the instance from Nova, and then cache them"},{"line_number":63,"context_line":"  to the global cache. Also, ceilometer-polling will record the resource"},{"line_number":64,"context_line":"  caching time to the cache system, and add an option named"},{"line_number":65,"context_line":"  \"interval_to_update_resource\" to indicate the max interval of resource cache"},{"line_number":66,"context_line":"  updating. If the time from last updating is larger than this option value,"},{"line_number":67,"context_line":"  the resource cache need to be updated again."}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a68dd71_beed1a27","line":64,"in_reply_to":"9a68dd71_b0efe71c","updated":"2016-01-27 02:26:08.000000000","message":"If we have cache updated from events, then the timestamp will be tagged to each instance, not the whole cache. How to deal with the situation of instance timestamp is not consistent with cache stamp?","commit_id":"43298c480a33e4945baeb6b2bfc4d92d425193eb"},{"author":{"_account_id":7478,"name":"Nadya Shakhat","email":"nadmi4@gmail.com","username":"nprivalova"},"change_message_id":"1e724eb07f14f841ce1470695b5bb6913d49e3f8","unresolved":false,"context_lines":[{"line_number":61,"context_line":"  resource metadata(instance info from Nova db). ceilometer-polling will use"},{"line_number":62,"context_line":"  \"instance_get_all\" to query all the instance from Nova, and then cache them"},{"line_number":63,"context_line":"  to the global cache. Also, ceilometer-polling will record the resource"},{"line_number":64,"context_line":"  caching time to the cache system, and add an option named"},{"line_number":65,"context_line":"  \"interval_to_update_resource\" to indicate the max interval of resource cache"},{"line_number":66,"context_line":"  updating. If the time from last updating is larger than this option value,"},{"line_number":67,"context_line":"  the resource cache need to be updated again."}],"source_content_type":"text/x-rst","patch_set":9,"id":"7a5de9d1_7337eeb0","line":64,"in_reply_to":"9a68dd71_beed1a27","updated":"2016-01-27 20:09:14.000000000","message":"cache stamp should not depend on updates from events. See the Gordon comment below. IIUC, the cache knows when it was updated last time. And when we validate the cache, we will look at this value only. Updates from events should not affect \"cache stamp\", these updates will help only to have more accurate data. So, assume that we updated the cache at 10 a.m. and have 1 hour \"interval_to_update_resource\", i.e. the next update should be at 11 a.m. If we get notification for some resource at 10.50, we will update the cache, but we will do one more update at 11 am anyway.","commit_id":"43298c480a33e4945baeb6b2bfc4d92d425193eb"},{"author":{"_account_id":6537,"name":"gordon chung","email":"gord@live.ca","username":"chungg"},"change_message_id":"27d773fdf62765d156b4c5b4dea08a57c5c908e9","unresolved":false,"context_lines":[{"line_number":63,"context_line":"  to the global cache. Also, ceilometer-polling will record the resource"},{"line_number":64,"context_line":"  caching time to the cache system, and add an option named"},{"line_number":65,"context_line":"  \"interval_to_update_resource\" to indicate the max interval of resource cache"},{"line_number":66,"context_line":"  updating. If the time from last updating is larger than this option value,"},{"line_number":67,"context_line":"  the resource cache need to be updated again."},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"  The \"change-since\" will also used as a query filter of \"instance_get_all\" to"},{"line_number":70,"context_line":"  massively reduce the amount of every query and only update the changed"}],"source_content_type":"text/x-rst","patch_set":9,"id":"7a5de9d1_f02a0de7","line":67,"range":{"start_line":66,"start_character":32,"end_line":67,"end_character":46},"updated":"2016-01-27 12:44:18.000000000","message":"can the compute agent handle everything? ie. during poll:\n\n1. check the cache\n2. verify that the cache is not stale (stale \u003d\u003d older than interval)\n2a if stale, lock and update cache (for all instances)\n2b if not stale, use the cache.\n3. get hypervisor?\n\nthe biggest concern might be the locking but tbh, it\u0027s polling so real-time isn\u0027t that big a deal as long as it\u0027s not a crazy long lock... this also avoids any need to coordinate task (ie. if you need to update cache, update it.)\n\nis this the idea being proposed?","commit_id":"43298c480a33e4945baeb6b2bfc4d92d425193eb"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"994fd2759819fe07db00c638e5c65129128aacb2","unresolved":false,"context_lines":[{"line_number":63,"context_line":"  to the global cache. Also, ceilometer-polling will record the resource"},{"line_number":64,"context_line":"  caching time to the cache system, and add an option named"},{"line_number":65,"context_line":"  \"interval_to_update_resource\" to indicate the max interval of resource cache"},{"line_number":66,"context_line":"  updating. If the time from last updating is larger than this option value,"},{"line_number":67,"context_line":"  the resource cache need to be updated again."},{"line_number":68,"context_line":""},{"line_number":69,"context_line":"  The \"change-since\" will also used as a query filter of \"instance_get_all\" to"},{"line_number":70,"context_line":"  massively reduce the amount of every query and only update the changed"}],"source_content_type":"text/x-rst","patch_set":9,"id":"7a5de9d1_7bbfbcd0","line":67,"range":{"start_line":66,"start_character":32,"end_line":67,"end_character":46},"in_reply_to":"7a5de9d1_f02a0de7","updated":"2016-01-28 02:57:46.000000000","message":"yes, you are right, this approach will minimize the change, this can also avoids the dependency on a centralized \"resource metadata polling\" every ceilometer-polling agent can perform the global resource updating action. for the potential lock issue, how about to shuffle the the startup time of all the polling agents to remit? Actually we have met a problem the the IO load will boost in every polling interval if all the polling agents running with a same interval, and I suggested them to shuffle the startup time (or polling interval?) of every polling agents, what do you think about ?","commit_id":"43298c480a33e4945baeb6b2bfc4d92d425193eb"},{"author":{"_account_id":7478,"name":"Nadya Shakhat","email":"nadmi4@gmail.com","username":"nprivalova"},"change_message_id":"3209bd1b135698616a588201d8f2ccb281111d1d","unresolved":false,"context_lines":[{"line_number":69,"context_line":"  The \"change-since\" will also used as a query filter of \"instance_get_all\" to"},{"line_number":70,"context_line":"  massively reduce the amount of every query and only update the changed"},{"line_number":71,"context_line":"  instance to the resource cache."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* In every ceilometer-polling of every compute node, unlike previous, agent"},{"line_number":74,"context_line":"  will try to query instances list of this host from the global resource cache"},{"line_number":75,"context_line":"  if the cached resources are not out of date, otherwise, query all the"}],"source_content_type":"text/x-rst","patch_set":9,"id":"9a68dd71_10cef362","line":72,"updated":"2016-01-25 13:38:52.000000000","message":"I like this approach. Liuseng, I remember that we talked about this and wanted to reduce the load on Rabbit as well. We would have been achieved that if would concatenate metadata to a sample at the notification agent level. But consider the situation when 3 days all notification agents are down (it may happen, you know:)). But user doesn\u0027t want to lose data. So, we switch on notification agents and start to apply a _new_ metadata to old samples. It will not work properly. To fix that, we need to have multiple copies of metadata in the cache, so implementation will be even more complex. \nThat\u0027s why I like the approach to have \"finished\" samples in the queue from the very beginning, it is safe. Yep, Rabbit is still overloaded, but let\u0027s consider this as a separate problem","commit_id":"43298c480a33e4945baeb6b2bfc4d92d425193eb"},{"author":{"_account_id":8290,"name":"liusheng","email":"liusheng2048@gmail.com","username":"LiuSheng"},"change_message_id":"994fd2759819fe07db00c638e5c65129128aacb2","unresolved":false,"context_lines":[{"line_number":69,"context_line":"  The \"change-since\" will also used as a query filter of \"instance_get_all\" to"},{"line_number":70,"context_line":"  massively reduce the amount of every query and only update the changed"},{"line_number":71,"context_line":"  instance to the resource cache."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"* In every ceilometer-polling of every compute node, unlike previous, agent"},{"line_number":74,"context_line":"  will try to query instances list of this host from the global resource cache"},{"line_number":75,"context_line":"  if the cached resources are not out of date, otherwise, query all the"}],"source_content_type":"text/x-rst","patch_set":9,"id":"7a5de9d1_60606736","line":72,"in_reply_to":"9a68dd71_10cef362","updated":"2016-01-28 02:57:46.000000000","message":"Hi Nadya,\nthanks for your explanation :), yeah, you are right! the previous approach have a big dependency on notification-agent, and that change may be more complex.\nAnd also, I think this approach is easy to switch off/on.","commit_id":"43298c480a33e4945baeb6b2bfc4d92d425193eb"}]}