)]}'
{"specs/rocky/approved/vgpu-rocky.rst":[{"author":{"_account_id":17920,"name":"Jianghua Wang","email":"jianghua.wang@citrix.com","username":"wjhfresh"},"change_message_id":"37e436f7902b62a4e3bd5618b940dae1a6616389","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"`Virtual GPUs in Nova`_ was implemented in Queens but only with one supported"},{"line_number":14,"context_line":"GPU type per compute node. Now that `Nested Resource Providers`_ is a thing,"},{"line_number":15,"context_line":"that spec is discussing about how to have one supported GPU type *per physical"},{"line_number":16,"context_line":"GPU*."},{"line_number":17,"context_line":""},{"line_number":18,"context_line":".. _`Virtual GPUs in Nova`: https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/add-support-for-vgpu.html"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_76c06c51","line":15,"range":{"start_line":15,"start_character":56,"end_line":15,"end_character":64},"updated":"2018-03-28 03:15:42.000000000","message":"vGPU type.","commit_id":"43c036e85d0c95fbe1c3710bdcb7e3eb77bf1d5c"},{"author":{"_account_id":17920,"name":"Jianghua Wang","email":"jianghua.wang@citrix.com","username":"wjhfresh"},"change_message_id":"37e436f7902b62a4e3bd5618b940dae1a6616389","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"`Virtual GPUs in Nova`_ was implemented in Queens but only with one supported"},{"line_number":14,"context_line":"GPU type per compute node. Now that `Nested Resource Providers`_ is a thing,"},{"line_number":15,"context_line":"that spec is discussing about how to have one supported GPU type *per physical"},{"line_number":16,"context_line":"GPU*."},{"line_number":17,"context_line":""},{"line_number":18,"context_line":".. _`Virtual GPUs in Nova`: https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/add-support-for-vgpu.html"},{"line_number":19,"context_line":".. _`Nested Resource Providers`: https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/nested-resource-providers.html"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_f6c79c44","line":16,"range":{"start_line":15,"start_character":70,"end_line":16,"end_character":3},"updated":"2018-03-28 03:15:42.000000000","message":"As you know Xen has same pGPUs in one group, it\u0027s \"one supported vGPU type per GPU group\". Can we follow the Queens\u0027 spec saying \"one vGPU type per RP for vGPUs\"?\n\nThe Queens\u0027 spec attempted to address the problem under N-RP, although finally we only support one VGPU type per compute node in Queens:\n\n\u003e\u003e\u003eSee the following paragraph copied from Queens\u0027 spec:\nAs described above, some pGPUs (e.g. NVIDIA GRID K1) support different sized vGPU types. The capacity for different vGPU types varies. In order to make resource tracking easier, we need to make sure the number of the vGPU is predictable. So we will add a new whitelist in nova.conf to specify the enabled vGPU types to ensure each resource provider of vGPUs only has one type of vGPUs.","commit_id":"43c036e85d0c95fbe1c3710bdcb7e3eb77bf1d5c"},{"author":{"_account_id":17920,"name":"Jianghua Wang","email":"jianghua.wang@citrix.com","username":"wjhfresh"},"change_message_id":"37e436f7902b62a4e3bd5618b940dae1a6616389","unresolved":false,"context_lines":[{"line_number":44,"context_line":"Proposed change"},{"line_number":45,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":46,"context_line":""},{"line_number":47,"context_line":"We already have ``[devices]/enabled_vgpu_types`` that define which types the"},{"line_number":48,"context_line":"Nova compute node can use:"},{"line_number":49,"context_line":""},{"line_number":50,"context_line":".. code::"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_f6847cb3","line":47,"range":{"start_line":47,"start_character":18,"end_line":47,"end_character":47},"updated":"2018-03-28 03:15:42.000000000","message":"For Xen, the same type of pGPUs will be put into one GPU group. And the vGPU types won\u0027t be same for different pGPU types. So I think the existing *eanbled_gpu_types* can work. What we need to support is the case that multiple pGPU types exist in single compute node (so there are multiple GPU groups).","commit_id":"43c036e85d0c95fbe1c3710bdcb7e3eb77bf1d5c"},{"author":{"_account_id":17920,"name":"Jianghua Wang","email":"jianghua.wang@citrix.com","username":"wjhfresh"},"change_message_id":"37e436f7902b62a4e3bd5618b940dae1a6616389","unresolved":false,"context_lines":[{"line_number":84,"context_line":".. code::"},{"line_number":85,"context_line":""},{"line_number":86,"context_line":"  [devices]"},{"line_number":87,"context_line":"  gpu_device_mapping \u003d {\"3d652a59-beaf-ddb3-3b19-c8c77ef60605\":\"GRID M60-0B\","},{"line_number":88,"context_line":"                        \"134a7b71-5ceb-8066-ef1b-3b319fb2bef3\":\"GRID P40-8A\"}"},{"line_number":89,"context_line":""},{"line_number":90,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_b1b4ce36","line":87,"range":{"start_line":87,"start_character":2,"end_line":87,"end_character":20},"updated":"2018-03-28 03:15:42.000000000","message":"Can we keep the existing option of *enabled_vgpu_types* but make this new option as virt specific? Different hypervisor may have different way to ensure each GPU RP only supply one type of vGPUs. e.g. As what I said above, I don\u0027t see any need for Xen to use this new option. Instead it will make things easier for operators if we only use *enabled_vgpu_types*.","commit_id":"43c036e85d0c95fbe1c3710bdcb7e3eb77bf1d5c"},{"author":{"_account_id":17920,"name":"Jianghua Wang","email":"jianghua.wang@citrix.com","username":"wjhfresh"},"change_message_id":"37e436f7902b62a4e3bd5618b940dae1a6616389","unresolved":false,"context_lines":[{"line_number":94,"context_line":"Alternatives"},{"line_number":95,"context_line":"------------"},{"line_number":96,"context_line":""},{"line_number":97,"context_line":"Instead of creating only one inventory per pGPU, we could try to have children"},{"line_number":98,"context_line":"resource providers for a pGPU being GPU types. But then, once an instance"},{"line_number":99,"context_line":"would create a vGPU, all the other inventories but the one related to the used"},{"line_number":100,"context_line":"type would be having a total\u003d0."}],"source_content_type":"text/x-rst","patch_set":1,"id":"bf659307_d1f942c8","line":97,"updated":"2018-03-28 03:15:42.000000000","message":"Another alternative we talked in Queens is to use a configure yaml file which can be used to specify which vGPU type is enabled in which PGPU. The yaml file can be pre-defined/changed by operators.","commit_id":"43c036e85d0c95fbe1c3710bdcb7e3eb77bf1d5c"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"49a9d22fe4ea41087965f9a1320620fc5e93c2ba","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"`Virtual GPUs in Nova`_ was implemented in Queens but only with one supported"},{"line_number":14,"context_line":"GPU type per compute node. Now that `Nested Resource Providers`_ is a thing,"},{"line_number":15,"context_line":"that spec is discussing about how to have one supported vGPU type *per physical"},{"line_number":16,"context_line":"GPU*."},{"line_number":17,"context_line":""},{"line_number":18,"context_line":".. note::"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_44330115","line":15,"range":{"start_line":15,"start_character":0,"end_line":15,"end_character":4},"updated":"2018-03-29 16:03:48.000000000","message":"this\n\nSaying \"that\" implies that it\u0027s the `Virtual GPUs in Nova` or `Nested Resource Providers` spec that\u0027s discussing...","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"49a9d22fe4ea41087965f9a1320620fc5e93c2ba","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"`Virtual GPUs in Nova`_ was implemented in Queens but only with one supported"},{"line_number":14,"context_line":"GPU type per compute node. Now that `Nested Resource Providers`_ is a thing,"},{"line_number":15,"context_line":"that spec is discussing about how to have one supported vGPU type *per physical"},{"line_number":16,"context_line":"GPU*."},{"line_number":17,"context_line":""},{"line_number":18,"context_line":".. note::"},{"line_number":19,"context_line":"   As Xen provides a specific feature where physical GPUs supporting same vGPU"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_642165b0","line":16,"range":{"start_line":15,"start_character":71,"end_line":16,"end_character":3},"updated":"2018-03-29 16:03:48.000000000","message":"As in \"GPU card\" or as in \"GPU physical function\"?","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"d86d36f6cdde05b32c6040bfb62df729589733e7","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"`Virtual GPUs in Nova`_ was implemented in Queens but only with one supported"},{"line_number":14,"context_line":"GPU type per compute node. Now that `Nested Resource Providers`_ is a thing,"},{"line_number":15,"context_line":"that spec is discussing about how to have one supported vGPU type *per physical"},{"line_number":16,"context_line":"GPU*."},{"line_number":17,"context_line":""},{"line_number":18,"context_line":".. note::"},{"line_number":19,"context_line":"   As Xen provides a specific feature where physical GPUs supporting same vGPU"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_a4a52b49","line":16,"range":{"start_line":15,"start_character":71,"end_line":16,"end_character":3},"in_reply_to":"bf659307_2b86d867","updated":"2018-03-30 09:34:03.000000000","message":"\u003e As in \"GPU card\" or as in \"GPU physical function\"?\n\nPer GPU physical function, as a single GPU card can have multiple PCI IDs.\n\n\u003e Where are they types defined? Meaning, the attributes of the type\n\nThose attributes are defined through a sysfs specific directory as defined in https://www.kernel.org/doc/Documentation/vfio-mediated-device.txt (\"\nDirectories and files under the sysfs for Each Physical Device\")\n\nEach physical GPU function is showing a list of different supported types. That said, once a mediated device is created, all the other types are still existing *but* their available_instances value is equal to 0, which is explained by http://docs.nvidia.com/grid/5.0/grid-vgpu-user-guide/index.html#homogeneous-grid-vgpus\n\nChoosing which type for which pGPU is not something libvirt or any management system is providing as of now (at least AFAIK). We *could* ask to pre-create a single mediated device for each pGPU so it would make that mapping, but that looks to me more difficult for a config management system to handle CLI calls (like litterally creating a new mdev) than just managing a conf option, but I\u0027m open to alternatives.","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"6611fd29b6ba148a1250fdf2d8711c9d26fe44fa","unresolved":false,"context_lines":[{"line_number":12,"context_line":""},{"line_number":13,"context_line":"`Virtual GPUs in Nova`_ was implemented in Queens but only with one supported"},{"line_number":14,"context_line":"GPU type per compute node. Now that `Nested Resource Providers`_ is a thing,"},{"line_number":15,"context_line":"that spec is discussing about how to have one supported vGPU type *per physical"},{"line_number":16,"context_line":"GPU*."},{"line_number":17,"context_line":""},{"line_number":18,"context_line":".. note::"},{"line_number":19,"context_line":"   As Xen provides a specific feature where physical GPUs supporting same vGPU"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_2b86d867","line":16,"range":{"start_line":15,"start_character":71,"end_line":16,"end_character":3},"in_reply_to":"bf659307_642165b0","updated":"2018-03-29 18:37:37.000000000","message":"Where are they types defined? Meaning, the attributes of the type, not just the name in our config file. Is it really our responsibility to make sure we don\u0027t spawn more than one type on a single card? I would expect that the types are defined somewhere on the system and already tied to the physical card, no?","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"0ee26b63fff3304f8da9388b4c645893bb33eae3","unresolved":false,"context_lines":[{"line_number":17,"context_line":""},{"line_number":18,"context_line":".. note::"},{"line_number":19,"context_line":"   As Xen provides a specific feature where physical GPUs supporting same vGPU"},{"line_number":20,"context_line":"   type are within a same pGPU group, that virt driver doesn\u0027t need to know"},{"line_number":21,"context_line":"   which exact pGPUs need to support a specific type, hence this spec only"},{"line_number":22,"context_line":"   targeted for the libvirt driver."},{"line_number":23,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"9f6a8fd7_b89f23b3","line":20,"range":{"start_line":20,"start_character":21,"end_line":20,"end_character":26},"updated":"2018-04-16 14:39:52.000000000","message":"s/same/single","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"0ee26b63fff3304f8da9388b4c645893bb33eae3","unresolved":false,"context_lines":[{"line_number":19,"context_line":"   As Xen provides a specific feature where physical GPUs supporting same vGPU"},{"line_number":20,"context_line":"   type are within a same pGPU group, that virt driver doesn\u0027t need to know"},{"line_number":21,"context_line":"   which exact pGPUs need to support a specific type, hence this spec only"},{"line_number":22,"context_line":"   targeted for the libvirt driver."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":".. _`Virtual GPUs in Nova`: https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/add-support-for-vgpu.html"},{"line_number":25,"context_line":".. _`Nested Resource Providers`: https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/nested-resource-providers.html"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9f6a8fd7_38ab13d7","line":22,"range":{"start_line":22,"start_character":3,"end_line":22,"end_character":11},"updated":"2018-04-16 14:39:52.000000000","message":"s/targeted for/targets/","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"0ee26b63fff3304f8da9388b4c645893bb33eae3","unresolved":false,"context_lines":[{"line_number":28,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":29,"context_line":""},{"line_number":30,"context_line":"Hardware vendors tell us that a physical GPU device can support multiple types."},{"line_number":31,"context_line":"That said, for homegeneous concerns, only one type can be used for all the"},{"line_number":32,"context_line":"virtual devices per graphical processing unit."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"For example, NVidia GRID physical cards can accept a list of different GPU"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9f6a8fd7_78f74bf7","line":31,"range":{"start_line":31,"start_character":9,"end_line":31,"end_character":37},"updated":"2018-04-16 14:39:52.000000000","message":"do you mean to say \"In order to keep deployments somewhat homogeneous, we intend to only support one vGPU type per physical GPU\"?","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"49a9d22fe4ea41087965f9a1320620fc5e93c2ba","unresolved":false,"context_lines":[{"line_number":40,"context_line":""},{"line_number":41,"context_line":"That\u0027s why we\u0027re missing at the moment a way to describe for each physical"},{"line_number":42,"context_line":"GPU which type it should support. Like we said above, just keep in mind that"},{"line_number":43,"context_line":"problem is only related to libvirt as of now (until other virt driver but Xen"},{"line_number":44,"context_line":"implements vGPU support)."},{"line_number":45,"context_line":""},{"line_number":46,"context_line":"Use Cases"},{"line_number":47,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_b73d6dc5","line":44,"range":{"start_line":43,"start_character":52,"end_line":44,"end_character":10},"updated":"2018-03-29 16:03:48.000000000","message":"virt drivers other than Xen implement","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"0ee26b63fff3304f8da9388b4c645893bb33eae3","unresolved":false,"context_lines":[{"line_number":75,"context_line":"                   A dict having a specific key/value format:"},{"line_number":76,"context_line":"                   {\"str_vgpu_type\": \"str_p_gpu\"}"},{"line_number":77,"context_line":"                   where the key is the VGPU type it will support"},{"line_number":78,"context_line":"                   and the value is a list of pGPU PCI addresses"},{"line_number":79,"context_line":"              \"\"\""},{"line_number":80,"context_line":""},{"line_number":81,"context_line":"For example, it would be set in nova.conf:"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9f6a8fd7_1873af82","line":78,"range":{"start_line":78,"start_character":37,"end_line":78,"end_character":64},"updated":"2018-04-16 14:39:52.000000000","message":"PCI addresses or mdev addresses/names?","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"49a9d22fe4ea41087965f9a1320620fc5e93c2ba","unresolved":false,"context_lines":[{"line_number":83,"context_line":".. code::"},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"  [libvirt]"},{"line_number":86,"context_line":"  vgpu_type_mapping \u003d {\"nvidia-35\":\"0000:84:00.0, 0000:85:00.0\","},{"line_number":87,"context_line":"                       \"nvidia-36\":\"0000:86:00.0\"}"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"In that case, the ``nvidia-35`` vGPU type would be supported by the physical"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_f76d25cc","line":86,"range":{"start_line":86,"start_character":35,"end_line":86,"end_character":63},"updated":"2018-03-29 16:03:48.000000000","message":"square brackets here","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"ac7bd3f815c4c5e5d3442f01d342e06274374205","unresolved":false,"context_lines":[{"line_number":83,"context_line":".. code::"},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"  [libvirt]"},{"line_number":86,"context_line":"  vgpu_type_mapping \u003d {\"nvidia-35\":\"0000:84:00.0, 0000:85:00.0\","},{"line_number":87,"context_line":"                       \"nvidia-36\":\"0000:86:00.0\"}"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"In that case, the ``nvidia-35`` vGPU type would be supported by the physical"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9f6a8fd7_0f298799","line":86,"range":{"start_line":86,"start_character":35,"end_line":86,"end_character":63},"in_reply_to":"9f6a8fd7_381eb30c","updated":"2018-04-23 15:36:40.000000000","message":"Could go the way of cinder, as dansmith has suggested elsewhere. Something like the following:\n\n  [compute]\n  vgpu_types \u003d nvidia35,nvidia36\n\n  [vgpu_nvidia35]\n  addresses \u003d 0000:84:00.0,0000:85:00.0\n\n  ...","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"0ee26b63fff3304f8da9388b4c645893bb33eae3","unresolved":false,"context_lines":[{"line_number":83,"context_line":".. code::"},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"  [libvirt]"},{"line_number":86,"context_line":"  vgpu_type_mapping \u003d {\"nvidia-35\":\"0000:84:00.0, 0000:85:00.0\","},{"line_number":87,"context_line":"                       \"nvidia-36\":\"0000:86:00.0\"}"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"In that case, the ``nvidia-35`` vGPU type would be supported by the physical"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9f6a8fd7_381eb30c","line":86,"range":{"start_line":86,"start_character":35,"end_line":86,"end_character":63},"in_reply_to":"bf659307_a428cb96","updated":"2018-04-16 14:39:52.000000000","message":"+1 on using MultiStrOpt. -1 on making the key the address instead of the vGPU type. It should be type: address IMHO.","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"d86d36f6cdde05b32c6040bfb62df729589733e7","unresolved":false,"context_lines":[{"line_number":83,"context_line":".. code::"},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"  [libvirt]"},{"line_number":86,"context_line":"  vgpu_type_mapping \u003d {\"nvidia-35\":\"0000:84:00.0, 0000:85:00.0\","},{"line_number":87,"context_line":"                       \"nvidia-36\":\"0000:86:00.0\"}"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"In that case, the ``nvidia-35`` vGPU type would be supported by the physical"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_a428cb96","line":86,"range":{"start_line":86,"start_character":35,"end_line":86,"end_character":63},"in_reply_to":"bf659307_cb687cba","updated":"2018-03-30 09:34:03.000000000","message":"Would it be more reasonable if we make it the other way around, like (as it\u0027s a MultiStrOpt) :\n\n vgpu_type_mapping \u003d {\"0000:84:00.0\": \"nvidia-35\"}\n vgpu_type_mapping \u003d {\"0000:85:00.0\": \"nvidia-35\"}\n vgpu_type_mapping \u003d {\"0000:86:00.0\": \"nvidia-36\"}\n\nThat would do the exact same thing where 2 pGPUs share the same type but a third one is using another type ?","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"6611fd29b6ba148a1250fdf2d8711c9d26fe44fa","unresolved":false,"context_lines":[{"line_number":83,"context_line":".. code::"},{"line_number":84,"context_line":""},{"line_number":85,"context_line":"  [libvirt]"},{"line_number":86,"context_line":"  vgpu_type_mapping \u003d {\"nvidia-35\":\"0000:84:00.0, 0000:85:00.0\","},{"line_number":87,"context_line":"                       \"nvidia-36\":\"0000:86:00.0\"}"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"In that case, the ``nvidia-35`` vGPU type would be supported by the physical"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_cb687cba","line":86,"range":{"start_line":86,"start_character":35,"end_line":86,"end_character":63},"in_reply_to":"bf659307_f76d25cc","updated":"2018-03-29 18:37:37.000000000","message":"This is exactly why I think this should not be a dict in string form in the config. Asking operators to write json by hand or from puppet is not reasonable, IMHO. We\u0027ve also had problems in the past with this format changing since people expect it\u0027s just free-form JSON, causing upgrade issues.","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"49a9d22fe4ea41087965f9a1320620fc5e93c2ba","unresolved":false,"context_lines":[{"line_number":84,"context_line":""},{"line_number":85,"context_line":"  [libvirt]"},{"line_number":86,"context_line":"  vgpu_type_mapping \u003d {\"nvidia-35\":\"0000:84:00.0, 0000:85:00.0\","},{"line_number":87,"context_line":"                       \"nvidia-36\":\"0000:86:00.0\"}"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"In that case, the ``nvidia-35`` vGPU type would be supported by the physical"},{"line_number":90,"context_line":"GPUs that are in the PCI addresses ``0000:84:00.0`` and ``0000:85:00.0``, while"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_b76b2db5","line":87,"range":{"start_line":87,"start_character":35,"end_line":87,"end_character":49},"updated":"2018-03-29 16:03:48.000000000","message":"and probably also here","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"49a9d22fe4ea41087965f9a1320620fc5e93c2ba","unresolved":false,"context_lines":[{"line_number":90,"context_line":"GPUs that are in the PCI addresses ``0000:84:00.0`` and ``0000:85:00.0``, while"},{"line_number":91,"context_line":"``nvidia-36`` vGPU would only be supported by ``0000:86:00.0``."},{"line_number":92,"context_line":""},{"line_number":93,"context_line":"If some operator messes up and provides two types for the same pGPU, only the"},{"line_number":94,"context_line":"first type will be taken in account. If the operator misses to provide a type"},{"line_number":95,"context_line":"for a specific pGPU, then the first type given in ``enabled_vgpu_types`` will"},{"line_number":96,"context_line":"be supported, like the existing situation."},{"line_number":97,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_37863d68","line":94,"range":{"start_line":93,"start_character":69,"end_line":94,"end_character":35},"updated":"2018-03-29 16:03:48.000000000","message":"or error, might be better.","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"0ee26b63fff3304f8da9388b4c645893bb33eae3","unresolved":false,"context_lines":[{"line_number":90,"context_line":"GPUs that are in the PCI addresses ``0000:84:00.0`` and ``0000:85:00.0``, while"},{"line_number":91,"context_line":"``nvidia-36`` vGPU would only be supported by ``0000:86:00.0``."},{"line_number":92,"context_line":""},{"line_number":93,"context_line":"If some operator messes up and provides two types for the same pGPU, only the"},{"line_number":94,"context_line":"first type will be taken in account. If the operator misses to provide a type"},{"line_number":95,"context_line":"for a specific pGPU, then the first type given in ``enabled_vgpu_types`` will"},{"line_number":96,"context_line":"be supported, like the existing situation."},{"line_number":97,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"9f6a8fd7_98bb3fe4","line":94,"range":{"start_line":93,"start_character":69,"end_line":94,"end_character":35},"in_reply_to":"bf659307_37863d68","updated":"2018-04-16 14:39:52.000000000","message":"Yes, please have the code throw an error on bad configuration.","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"6611fd29b6ba148a1250fdf2d8711c9d26fe44fa","unresolved":false,"context_lines":[{"line_number":93,"context_line":"If some operator messes up and provides two types for the same pGPU, only the"},{"line_number":94,"context_line":"first type will be taken in account. If the operator misses to provide a type"},{"line_number":95,"context_line":"for a specific pGPU, then the first type given in ``enabled_vgpu_types`` will"},{"line_number":96,"context_line":"be supported, like the existing situation."},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"Alternatives"},{"line_number":99,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_cb3f1cb2","line":96,"updated":"2018-03-29 18:37:37.000000000","message":"And what if they fat-finger the format?","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"d86d36f6cdde05b32c6040bfb62df729589733e7","unresolved":false,"context_lines":[{"line_number":93,"context_line":"If some operator messes up and provides two types for the same pGPU, only the"},{"line_number":94,"context_line":"first type will be taken in account. If the operator misses to provide a type"},{"line_number":95,"context_line":"for a specific pGPU, then the first type given in ``enabled_vgpu_types`` will"},{"line_number":96,"context_line":"be supported, like the existing situation."},{"line_number":97,"context_line":""},{"line_number":98,"context_line":"Alternatives"},{"line_number":99,"context_line":"------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_a401ab08","line":96,"in_reply_to":"bf659307_cb3f1cb2","updated":"2018-03-30 09:34:03.000000000","message":"Good point, I guess we could make nova-compute refusing to start if they mess up their config options (like in the driver init_host), like by fat-fingering the GPU types.","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"49a9d22fe4ea41087965f9a1320620fc5e93c2ba","unresolved":false,"context_lines":[{"line_number":105,"context_line":"Also, doing that could be a race condition given the Placement allocation would"},{"line_number":106,"context_line":"be done against a specific GPU type but the inventories for the other types"},{"line_number":107,"context_line":"would still be accepting other vGPUs until the compute service creates the"},{"line_number":108,"context_line":"original vGPU."},{"line_number":109,"context_line":""},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"Data model impact"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_17f699fa","line":108,"updated":"2018-03-29 16:03:48.000000000","message":"++","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"0ee26b63fff3304f8da9388b4c645893bb33eae3","unresolved":false,"context_lines":[{"line_number":102,"context_line":"resource providers for a pGPU being GPU types. But then, once an instance"},{"line_number":103,"context_line":"would create a vGPU, all the other inventories but the one related to the used"},{"line_number":104,"context_line":"type would be having a total\u003d0."},{"line_number":105,"context_line":"Also, doing that could be a race condition given the Placement allocation would"},{"line_number":106,"context_line":"be done against a specific GPU type but the inventories for the other types"},{"line_number":107,"context_line":"would still be accepting other vGPUs until the compute service creates the"},{"line_number":108,"context_line":"original vGPU."},{"line_number":109,"context_line":""},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"Data model impact"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9f6a8fd7_58836705","line":108,"range":{"start_line":105,"start_character":0,"end_line":108,"end_character":14},"updated":"2018-04-16 14:39:52.000000000","message":"I don\u0027t understand what you mean by this sentence.","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":14070,"name":"Eric Fried","email":"openstack@fried.cc","username":"efried"},"change_message_id":"49a9d22fe4ea41087965f9a1320620fc5e93c2ba","unresolved":false,"context_lines":[{"line_number":106,"context_line":"be done against a specific GPU type but the inventories for the other types"},{"line_number":107,"context_line":"would still be accepting other vGPUs until the compute service creates the"},{"line_number":108,"context_line":"original vGPU."},{"line_number":109,"context_line":""},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"Data model impact"},{"line_number":112,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_37071dcd","line":109,"updated":"2018-03-29 16:03:48.000000000","message":"There are alternatives to using a [libvirt] conf option.  Eventually I hope this kind of information is subsumed into whatever separate config file we invent to handle generic device management.  It may be premature to try to design that here; but if/when that does develop, hopefully this proposed conf would be deprecated in favor of it.","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":7166,"name":"Sylvain Bauza","email":"sbauza@redhat.com","username":"sbauza"},"change_message_id":"d86d36f6cdde05b32c6040bfb62df729589733e7","unresolved":false,"context_lines":[{"line_number":106,"context_line":"be done against a specific GPU type but the inventories for the other types"},{"line_number":107,"context_line":"would still be accepting other vGPUs until the compute service creates the"},{"line_number":108,"context_line":"original vGPU."},{"line_number":109,"context_line":""},{"line_number":110,"context_line":""},{"line_number":111,"context_line":"Data model impact"},{"line_number":112,"context_line":"-----------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"bf659307_df13485b","line":109,"in_reply_to":"bf659307_37071dcd","updated":"2018-03-30 09:34:03.000000000","message":"Yeah, a YAML file describing inventories (like Jay started to spec up in https://review.openstack.org/#/c/550244/)\n\nThat\u0027s also why I think a conf option is a reasonable trade-off as we can easily deprecate it once a better approach is made.","commit_id":"f8da147b73394017bac193a4577d0336800a2720"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"9e2803a4550e9bbd9cf3530ee856c63f08e7004e","unresolved":false,"context_lines":[{"line_number":5,"context_line":" http://creativecommons.org/licenses/by/3.0/legalcode"},{"line_number":6,"context_line":""},{"line_number":7,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":8,"context_line":"Supporting multiple GPU types for a single pGPU"},{"line_number":9,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":10,"context_line":""},{"line_number":11,"context_line":"https://blueprints.launchpad.net/nova/+spec/vgpu-rocky"}],"source_content_type":"text/x-rst","patch_set":3,"id":"5f7c97a3_b9b45281","line":8,"range":{"start_line":8,"start_character":20,"end_line":8,"end_character":23},"updated":"2018-05-14 18:50:22.000000000","message":"vGPU types...","commit_id":"28a06e039b8e5758bbf154d70c9991397b6f77a6"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"9e2803a4550e9bbd9cf3530ee856c63f08e7004e","unresolved":false,"context_lines":[{"line_number":101,"context_line":"``nvidia-36`` vGPU would only be supported by ``0000:86:00.0``."},{"line_number":102,"context_line":""},{"line_number":103,"context_line":"If some operator messes up and provides two types for the same pGPU, an"},{"line_number":104,"context_line":"InvalidLibvirtGPUConfig exception will be raised. If the operator misses to"},{"line_number":105,"context_line":"provide a type for a specific pGPU, then the first type given in"},{"line_number":106,"context_line":"``enabled_vgpu_types`` will be supported, like the existing situation."},{"line_number":107,"context_line":"If the operator fat-fingers the PCI IDs, then when creating the inventory, it"}],"source_content_type":"text/x-rst","patch_set":3,"id":"5f7c97a3_f9cf2a26","line":104,"range":{"start_line":104,"start_character":66,"end_line":104,"end_character":72},"updated":"2018-05-14 18:50:22.000000000","message":"s/misses/forgets/","commit_id":"28a06e039b8e5758bbf154d70c9991397b6f77a6"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"d8a565d89d01078fb498029d25c580ea7c171084","unresolved":false,"context_lines":[{"line_number":18,"context_line":".. note::"},{"line_number":19,"context_line":"   As Xen provides a specific feature where physical GPUs supporting same vGPU"},{"line_number":20,"context_line":"   type are within a single pGPU group, that virt driver doesn\u0027t need to know"},{"line_number":21,"context_line":"   which exact pGPUs need to support a specific type, hence this spec only"},{"line_number":22,"context_line":"   targets the libvirt driver."},{"line_number":23,"context_line":""},{"line_number":24,"context_line":".. _`Virtual GPUs in Nova`: https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/add-support-for-vgpu.html"},{"line_number":25,"context_line":".. _`Nested Resource Providers`: https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/nested-resource-providers.html"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_ba25125e","line":22,"range":{"start_line":21,"start_character":54,"end_line":22,"end_character":30},"updated":"2018-05-31 16:41:26.000000000","message":"and therefore the spec/blueprint name should be prefixed with \"libvirt-\".","commit_id":"8363b4a7fe46b7a5cc580e8cb92f4150634c21d0"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"d8a565d89d01078fb498029d25c580ea7c171084","unresolved":false,"context_lines":[{"line_number":28,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":29,"context_line":""},{"line_number":30,"context_line":"Hardware vendors tell us that a physical GPU device can support multiple types."},{"line_number":31,"context_line":"That said, most of the vendor drivers only accepts one type for all the"},{"line_number":32,"context_line":"virtual devices *per graphical processing unit*."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"For example, NVidia GRID physical cards can accept a list of different GPU"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_5a14fee8","line":31,"range":{"start_line":31,"start_character":43,"end_line":31,"end_character":50},"updated":"2018-05-31 16:41:26.000000000","message":"accept","commit_id":"8363b4a7fe46b7a5cc580e8cb92f4150634c21d0"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"d8a565d89d01078fb498029d25c580ea7c171084","unresolved":false,"context_lines":[{"line_number":28,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":29,"context_line":""},{"line_number":30,"context_line":"Hardware vendors tell us that a physical GPU device can support multiple types."},{"line_number":31,"context_line":"That said, most of the vendor drivers only accepts one type for all the"},{"line_number":32,"context_line":"virtual devices *per graphical processing unit*."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"For example, NVidia GRID physical cards can accept a list of different GPU"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_1ad8c650","line":31,"range":{"start_line":31,"start_character":11,"end_line":31,"end_character":37},"updated":"2018-05-31 16:41:26.000000000","message":"it\u0027s not \"most of the vendor drivers\". it\u0027s NVIDIA. Let\u0027s at least be honest here.\n\nIntel\u0027s GVT-g technology doesn\u0027t have \"vGPU types\". This is all about NVIDIA, and only NVIDIA.","commit_id":"8363b4a7fe46b7a5cc580e8cb92f4150634c21d0"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"d8a565d89d01078fb498029d25c580ea7c171084","unresolved":false,"context_lines":[{"line_number":38,"context_line":""},{"line_number":39,"context_line":".. _`one type per physical GPU`: http://docs.nvidia.com/grid/5.0/grid-vgpu-user-guide/index.html#homogeneous-grid-vgpus"},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"That\u0027s why we\u0027re missing at the moment a way to describe for each physical"},{"line_number":42,"context_line":"GPU which type it should support. Like we said above, just keep in mind that"},{"line_number":43,"context_line":"problem is only related to libvirt as of now (until virt drivers other than Xen"},{"line_number":44,"context_line":"implement vGPU support)."},{"line_number":45,"context_line":""},{"line_number":46,"context_line":"Use Cases"},{"line_number":47,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_3ac3423a","line":44,"range":{"start_line":41,"start_character":0,"end_line":44,"end_character":24},"updated":"2018-05-31 16:41:26.000000000","message":"Suggest deleting this paragraph. Describe the problem domain instead. Something like the following would be an honest, accurate description of the problem:\n\n We require a way to instruct the libvirt driver which NVIDIA\n vGPU types an NVIDIA physical GPU is configured to accept.","commit_id":"8363b4a7fe46b7a5cc580e8cb92f4150634c21d0"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"d8a565d89d01078fb498029d25c580ea7c171084","unresolved":false,"context_lines":[{"line_number":46,"context_line":"Use Cases"},{"line_number":47,"context_line":"---------"},{"line_number":48,"context_line":""},{"line_number":49,"context_line":"As an operator, I\u0027d like to propose different GPU types for my users, by saying"},{"line_number":50,"context_line":"which type each physical GPU would support."},{"line_number":51,"context_line":""},{"line_number":52,"context_line":"Proposed change"},{"line_number":53,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_7ad21a02","line":50,"range":{"start_line":49,"start_character":0,"end_line":50,"end_character":43},"updated":"2018-05-31 16:41:26.000000000","message":"This isn\u0027t a use case, IMHO. I\u0027d leave the use case as \"An operator needs a way to inform the libvirt driver which NVIDIA vGPU types an NVIDIA physical GPU is configured to accept.\"","commit_id":"8363b4a7fe46b7a5cc580e8cb92f4150634c21d0"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"d8a565d89d01078fb498029d25c580ea7c171084","unresolved":false,"context_lines":[{"line_number":60,"context_line":"  [devices]"},{"line_number":61,"context_line":"  enabled_vgpu_types \u003d [str_vgpu_type_1, str_vgpu_type_2, ...]"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"Now we propose that libvirt (given for the moment it\u0027s only supported by that"},{"line_number":64,"context_line":"virt driver) will dynamically accept configuration sections that are related to"},{"line_number":65,"context_line":"the [devices]/enabled_vgpu_types and specifies which exact pGPUs are related"},{"line_number":66,"context_line":"to the enabled vGPU types and will have a ``device_addresses`` option defined"},{"line_number":67,"context_line":"like this:"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_5ad39e01","line":64,"range":{"start_line":63,"start_character":28,"end_line":64,"end_character":13},"updated":"2018-05-31 16:41:26.000000000","message":"strike","commit_id":"8363b4a7fe46b7a5cc580e8cb92f4150634c21d0"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"d8a565d89d01078fb498029d25c580ea7c171084","unresolved":false,"context_lines":[{"line_number":61,"context_line":"  enabled_vgpu_types \u003d [str_vgpu_type_1, str_vgpu_type_2, ...]"},{"line_number":62,"context_line":""},{"line_number":63,"context_line":"Now we propose that libvirt (given for the moment it\u0027s only supported by that"},{"line_number":64,"context_line":"virt driver) will dynamically accept configuration sections that are related to"},{"line_number":65,"context_line":"the [devices]/enabled_vgpu_types and specifies which exact pGPUs are related"},{"line_number":66,"context_line":"to the enabled vGPU types and will have a ``device_addresses`` option defined"},{"line_number":67,"context_line":"like this:"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_5aecfeb9","line":64,"range":{"start_line":64,"start_character":18,"end_line":64,"end_character":29},"updated":"2018-05-31 16:41:26.000000000","message":"what do you mean specifically by \"dynamically\"? It\u0027s not as if you can change the accepted vGPU type for an NVIDIA physical GPU once that pGPU has had a vGPU of that type created on it.","commit_id":"8363b4a7fe46b7a5cc580e8cb92f4150634c21d0"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"d8a565d89d01078fb498029d25c580ea7c171084","unresolved":false,"context_lines":[{"line_number":73,"context_line":"              help\u003d\"\"\""},{"line_number":74,"context_line":"  List of physical PCI addresses to associate with a specific GPU type."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"  The GPU type to associate with the PCI devices has to be the section name"},{"line_number":77,"context_line":"  prefixed by ``gpu_``. For example, for \u0027nvidia-11\u0027, you would declare"},{"line_number":78,"context_line":"  ``[gpu_nvidia-11]/device_addresses``."},{"line_number":79,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_1a7e86e3","line":76,"range":{"start_line":76,"start_character":37,"end_line":76,"end_character":48},"updated":"2018-05-31 16:41:26.000000000","message":"to make this more clear, I\u0027d put something like \"The particular NVIDIA physical GPU device address needs to be mapped to the NVIDIA vGPU type which that physical GPU is configured to accept. In order to provide this mapping, there will be a CONF section with a name corresponding to the following template: \"vgpu_type_%(vgpu_type_name)s\". This section will have a single CONF option called device_addresses which contain the addresses (PCI, mediated, etc) of the NVIDIA physical devices that accept the corresponding NVIDIA vGPU type.","commit_id":"8363b4a7fe46b7a5cc580e8cb92f4150634c21d0"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"d8a565d89d01078fb498029d25c580ea7c171084","unresolved":false,"context_lines":[{"line_number":73,"context_line":"              help\u003d\"\"\""},{"line_number":74,"context_line":"  List of physical PCI addresses to associate with a specific GPU type."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"  The GPU type to associate with the PCI devices has to be the section name"},{"line_number":77,"context_line":"  prefixed by ``gpu_``. For example, for \u0027nvidia-11\u0027, you would declare"},{"line_number":78,"context_line":"  ``[gpu_nvidia-11]/device_addresses``."},{"line_number":79,"context_line":""}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_da832eca","line":76,"range":{"start_line":76,"start_character":6,"end_line":76,"end_character":9},"updated":"2018-05-31 16:41:26.000000000","message":"vGPU type (to be consistent with \"enabled_vgpu_types\"). It\u0027s too bad we didn\u0027t insist on calling this nvidia_enabled_vgpu_types, since this is all entirely specific to NVIDIA.","commit_id":"8363b4a7fe46b7a5cc580e8cb92f4150634c21d0"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"d8a565d89d01078fb498029d25c580ea7c171084","unresolved":false,"context_lines":[{"line_number":74,"context_line":"  List of physical PCI addresses to associate with a specific GPU type."},{"line_number":75,"context_line":""},{"line_number":76,"context_line":"  The GPU type to associate with the PCI devices has to be the section name"},{"line_number":77,"context_line":"  prefixed by ``gpu_``. For example, for \u0027nvidia-11\u0027, you would declare"},{"line_number":78,"context_line":"  ``[gpu_nvidia-11]/device_addresses``."},{"line_number":79,"context_line":""},{"line_number":80,"context_line":"  Each GPU type also has to be declared in ``[devices]/enabled_vgpu_types``."}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_5a6f1e27","line":77,"range":{"start_line":77,"start_character":16,"end_line":77,"end_character":20},"updated":"2018-05-31 16:41:26.000000000","message":"vgpu_","commit_id":"8363b4a7fe46b7a5cc580e8cb92f4150634c21d0"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"d8a565d89d01078fb498029d25c580ea7c171084","unresolved":false,"context_lines":[{"line_number":108,"context_line":"will return an exception."},{"line_number":109,"context_line":""},{"line_number":110,"context_line":".. note::"},{"line_number":111,"context_line":"   Given GPU capabilities can be very vendor-specific, it\u0027s a NP-problem for"},{"line_number":112,"context_line":"   Nova to guess which abstraction could be done for telling about GPU types."},{"line_number":113,"context_line":"   For that reason, the only way to not standardize vendor-specific knobs is"},{"line_number":114,"context_line":"   to ask operators to add custom traits directly to the nested Resource"},{"line_number":115,"context_line":"   Provider related to the GPU type by using the os-placement CLI."}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_9a5cd64b","line":112,"range":{"start_line":111,"start_character":55,"end_line":112,"end_character":77},"updated":"2018-05-31 16:41:26.000000000","message":"An \"NP-problem\"? No it\u0027s not. It\u0027s actually a *fixed-cost problem* (deterministic) to solve. Basically, you look up the set of fixed traits/capabilities that correspond to a particular NVIDIA vGPU type in the capabilities table for vGPU types in the NVIDIA product documentation:\n\nhttp://images.nvidia.com/content/grid/pdf/GRID-vGPU-User-Guide.pdf\n\nIn other words, it\u0027s an O(1) problem since it\u0027s a fixed hash-table lookup.","commit_id":"8363b4a7fe46b7a5cc580e8cb92f4150634c21d0"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"d8a565d89d01078fb498029d25c580ea7c171084","unresolved":false,"context_lines":[{"line_number":113,"context_line":"   For that reason, the only way to not standardize vendor-specific knobs is"},{"line_number":114,"context_line":"   to ask operators to add custom traits directly to the nested Resource"},{"line_number":115,"context_line":"   Provider related to the GPU type by using the os-placement CLI."},{"line_number":116,"context_line":"   Given virt drivers could also pass traits to Placement API, we suggest"},{"line_number":117,"context_line":"   operators to use namespaces like ``CUSTOM_NVIDIA_``."},{"line_number":118,"context_line":""},{"line_number":119,"context_line":""},{"line_number":120,"context_line":"Alternatives"}],"source_content_type":"text/x-rst","patch_set":5,"id":"5f7c97a3_5a52be36","line":117,"range":{"start_line":116,"start_character":3,"end_line":117,"end_character":55},"updated":"2018-05-31 16:41:26.000000000","message":"highly disagree here. These aren\u0027t actually custom traits. They are standard traits in os-traits. If we need more, we should add them there.","commit_id":"8363b4a7fe46b7a5cc580e8cb92f4150634c21d0"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"0d9cc4757bd6bf95f26a29e37f80850ca191b4b9","unresolved":false,"context_lines":[{"line_number":17,"context_line":""},{"line_number":18,"context_line":".. note::"},{"line_number":19,"context_line":"   As Xen provides a specific feature where physical GPUs supporting same vGPU"},{"line_number":20,"context_line":"   type are within a single pGPU group, that virt driver doesn\u0027t need to know"},{"line_number":21,"context_line":"   which exact pGPUs need to support a specific type, hence this spec only"},{"line_number":22,"context_line":"   targets the libvirt driver."},{"line_number":23,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_a53e0589","line":20,"range":{"start_line":20,"start_character":21,"end_line":20,"end_character":38},"updated":"2018-06-05 13:39:33.000000000","message":"I have no idea what a pGPU group is. Got docs?","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6f7f4bc03e0818af11ae1b5ac2882f2edaa1977b","unresolved":false,"context_lines":[{"line_number":17,"context_line":""},{"line_number":18,"context_line":".. note::"},{"line_number":19,"context_line":"   As Xen provides a specific feature where physical GPUs supporting same vGPU"},{"line_number":20,"context_line":"   type are within a single pGPU group, that virt driver doesn\u0027t need to know"},{"line_number":21,"context_line":"   which exact pGPUs need to support a specific type, hence this spec only"},{"line_number":22,"context_line":"   targets the libvirt driver."},{"line_number":23,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_358c43d6","line":20,"range":{"start_line":20,"start_character":21,"end_line":20,"end_character":38},"in_reply_to":"5f7c97a3_a53e0589","updated":"2018-06-06 18:34:42.000000000","message":"A pGPU group (or just GPU group) is described in the links below. XenAPI manages multiple physical GPUs using the concept of a pGPU group:\n\nSee Chapter 5 in NVIDIA\u0027s product documentation on XenServer VGPU management:\n\nhttps://docs.nvidia.com/grid/3.2/pdf/grid-vgpu-user-guide.pdf","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"0d9cc4757bd6bf95f26a29e37f80850ca191b4b9","unresolved":false,"context_lines":[{"line_number":28,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"},{"line_number":29,"context_line":""},{"line_number":30,"context_line":"Hardware vendors tell us that a physical GPU device can support multiple types."},{"line_number":31,"context_line":"That said, Intel (to be confirmed) and NVidia vendor drivers only accept one"},{"line_number":32,"context_line":"type for all the virtual devices *per graphical processing unit*."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"For example, NVidia GRID physical cards can accept a list of different GPU"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_853bc9c6","line":31,"range":{"start_line":31,"start_character":39,"end_line":31,"end_character":45},"updated":"2018-06-05 13:39:33.000000000","message":"nit: NVIDIA (right?)","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"0d9cc4757bd6bf95f26a29e37f80850ca191b4b9","unresolved":false,"context_lines":[{"line_number":29,"context_line":""},{"line_number":30,"context_line":"Hardware vendors tell us that a physical GPU device can support multiple types."},{"line_number":31,"context_line":"That said, Intel (to be confirmed) and NVidia vendor drivers only accept one"},{"line_number":32,"context_line":"type for all the virtual devices *per graphical processing unit*."},{"line_number":33,"context_line":""},{"line_number":34,"context_line":"For example, NVidia GRID physical cards can accept a list of different GPU"},{"line_number":35,"context_line":"types, but the driver can only support `one type per physical GPU`_."}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_e525ddaf","line":32,"range":{"start_line":32,"start_character":38,"end_line":32,"end_character":62},"updated":"2018-06-05 13:39:33.000000000","message":"nit: per GPU (if you\u0027re going to expand out GPU, do it earlier than here)","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"0d9cc4757bd6bf95f26a29e37f80850ca191b4b9","unresolved":false,"context_lines":[{"line_number":34,"context_line":"For example, NVidia GRID physical cards can accept a list of different GPU"},{"line_number":35,"context_line":"types, but the driver can only support `one type per physical GPU`_."},{"line_number":36,"context_line":""},{"line_number":37,"context_line":".. figure:: http://docs.nvidia.com/grid/5.0/grid-vgpu-user-guide/graphics/sample-vgpu-configurations-grid-2gpus-on-card.png"},{"line_number":38,"context_line":""},{"line_number":39,"context_line":".. _`one type per physical GPU`: http://docs.nvidia.com/grid/5.0/grid-vgpu-user-guide/index.html#homogeneous-grid-vgpus"},{"line_number":40,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_653eadba","line":37,"range":{"start_line":37,"start_character":12,"end_line":37,"end_character":123},"updated":"2018-06-05 13:39:33.000000000","message":"I wonder if it would be better to drag this in-tree and include a link to the original? That ensures we won\u0027t lose this if/when nvidia inevitably update their site and break our links","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6f7f4bc03e0818af11ae1b5ac2882f2edaa1977b","unresolved":false,"context_lines":[{"line_number":39,"context_line":".. _`one type per physical GPU`: http://docs.nvidia.com/grid/5.0/grid-vgpu-user-guide/index.html#homogeneous-grid-vgpus"},{"line_number":40,"context_line":""},{"line_number":41,"context_line":"Consequently, we require a way to instruct the libvirt driver which vGPU types"},{"line_number":42,"context_line":"an NVIDIA or Intel physical GPU is configured to accept."},{"line_number":43,"context_line":""},{"line_number":44,"context_line":"Use Cases"},{"line_number":45,"context_line":"---------"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_956bd7ef","line":42,"range":{"start_line":42,"start_character":3,"end_line":42,"end_character":31},"updated":"2018-06-06 18:34:42.000000000","message":"heh. It\u0027s really just NVIDIA that does this vgpu type stuff... Intel\u0027s GVT-g platform management interface can specify a vGPU type, but it\u0027s still just the NVIDIA product model. Plus, AMD also makes GPUs :)\n\nIn any case, I\u0027m not -1\u0027ing for this.","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"0d9cc4757bd6bf95f26a29e37f80850ca191b4b9","unresolved":false,"context_lines":[{"line_number":58,"context_line":"  [devices]"},{"line_number":59,"context_line":"  enabled_vgpu_types \u003d [str_vgpu_type_1, str_vgpu_type_2, ...]"},{"line_number":60,"context_line":""},{"line_number":61,"context_line":"Now we propose that libvirt will accept configuration sections that are related"},{"line_number":62,"context_line":"to the [devices]/enabled_vgpu_types and specifies which exact pGPUs are related"},{"line_number":63,"context_line":"to the enabled vGPU types and will have a ``device_addresses`` option defined"},{"line_number":64,"context_line":"like this:"},{"line_number":65,"context_line":""},{"line_number":66,"context_line":".. code::"},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"  cfg.ListOpt(\u0027device_addresses\u0027,"},{"line_number":69,"context_line":"              default\u003d[],"},{"line_number":70,"context_line":"              help\u003d\"\"\""},{"line_number":71,"context_line":"  List of physical PCI addresses to associate with a specific GPU type."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"  The particular physical GPU device address needs to be mapped to the vendor"},{"line_number":74,"context_line":"  vGPU type which that physical GPU is configured to accept. In order to"},{"line_number":75,"context_line":"  provide this mapping, there will be a CONF section with a name corresponding"},{"line_number":76,"context_line":"  to the following template: \"vgpu_type_%(vgpu_type_name)s"},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"  The vGPU type to associate with the PCI devices has to be the section name"},{"line_number":79,"context_line":"  prefixed by ``vgpu_``. For example, for \u0027nvidia-11\u0027, you would declare"},{"line_number":80,"context_line":"  ``[vgpu_nvidia-11]/device_addresses``."},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"  Each vGPU type also has to be declared in ``[devices]/enabled_vgpu_types``."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"  Related options:"},{"line_number":85,"context_line":""},{"line_number":86,"context_line":"  * ``[devices]/enabled_vgpu_types``"},{"line_number":87,"context_line":"  \"\"\"),"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"For example, it would be set in nova.conf:"},{"line_number":90,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_85a989de","line":87,"range":{"start_line":61,"start_character":0,"end_line":87,"end_character":7},"updated":"2018-06-05 13:39:33.000000000","message":"nit: I\u0027d drop this section and focus on how it would be configured in nova.conf. That\u0027s what I, as a reader of the spec, really care about.","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6f7f4bc03e0818af11ae1b5ac2882f2edaa1977b","unresolved":false,"context_lines":[{"line_number":58,"context_line":"  [devices]"},{"line_number":59,"context_line":"  enabled_vgpu_types \u003d [str_vgpu_type_1, str_vgpu_type_2, ...]"},{"line_number":60,"context_line":""},{"line_number":61,"context_line":"Now we propose that libvirt will accept configuration sections that are related"},{"line_number":62,"context_line":"to the [devices]/enabled_vgpu_types and specifies which exact pGPUs are related"},{"line_number":63,"context_line":"to the enabled vGPU types and will have a ``device_addresses`` option defined"},{"line_number":64,"context_line":"like this:"},{"line_number":65,"context_line":""},{"line_number":66,"context_line":".. code::"},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"  cfg.ListOpt(\u0027device_addresses\u0027,"},{"line_number":69,"context_line":"              default\u003d[],"},{"line_number":70,"context_line":"              help\u003d\"\"\""},{"line_number":71,"context_line":"  List of physical PCI addresses to associate with a specific GPU type."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"  The particular physical GPU device address needs to be mapped to the vendor"},{"line_number":74,"context_line":"  vGPU type which that physical GPU is configured to accept. In order to"},{"line_number":75,"context_line":"  provide this mapping, there will be a CONF section with a name corresponding"},{"line_number":76,"context_line":"  to the following template: \"vgpu_type_%(vgpu_type_name)s"},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"  The vGPU type to associate with the PCI devices has to be the section name"},{"line_number":79,"context_line":"  prefixed by ``vgpu_``. For example, for \u0027nvidia-11\u0027, you would declare"},{"line_number":80,"context_line":"  ``[vgpu_nvidia-11]/device_addresses``."},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"  Each vGPU type also has to be declared in ``[devices]/enabled_vgpu_types``."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"  Related options:"},{"line_number":85,"context_line":""},{"line_number":86,"context_line":"  * ``[devices]/enabled_vgpu_types``"},{"line_number":87,"context_line":"  \"\"\"),"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"For example, it would be set in nova.conf:"},{"line_number":90,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_f5324b0f","line":87,"range":{"start_line":61,"start_character":0,"end_line":87,"end_character":7},"in_reply_to":"5f7c97a3_85a989de","updated":"2018-06-06 18:34:42.000000000","message":"agree with @stephenfin","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":782,"name":"John Garbutt","email":"john@johngarbutt.com","username":"johngarbutt"},"change_message_id":"01cec7e7c21b8faec1de34f6bfa99242103fa724","unresolved":false,"context_lines":[{"line_number":58,"context_line":"  [devices]"},{"line_number":59,"context_line":"  enabled_vgpu_types \u003d [str_vgpu_type_1, str_vgpu_type_2, ...]"},{"line_number":60,"context_line":""},{"line_number":61,"context_line":"Now we propose that libvirt will accept configuration sections that are related"},{"line_number":62,"context_line":"to the [devices]/enabled_vgpu_types and specifies which exact pGPUs are related"},{"line_number":63,"context_line":"to the enabled vGPU types and will have a ``device_addresses`` option defined"},{"line_number":64,"context_line":"like this:"},{"line_number":65,"context_line":""},{"line_number":66,"context_line":".. code::"},{"line_number":67,"context_line":""},{"line_number":68,"context_line":"  cfg.ListOpt(\u0027device_addresses\u0027,"},{"line_number":69,"context_line":"              default\u003d[],"},{"line_number":70,"context_line":"              help\u003d\"\"\""},{"line_number":71,"context_line":"  List of physical PCI addresses to associate with a specific GPU type."},{"line_number":72,"context_line":""},{"line_number":73,"context_line":"  The particular physical GPU device address needs to be mapped to the vendor"},{"line_number":74,"context_line":"  vGPU type which that physical GPU is configured to accept. In order to"},{"line_number":75,"context_line":"  provide this mapping, there will be a CONF section with a name corresponding"},{"line_number":76,"context_line":"  to the following template: \"vgpu_type_%(vgpu_type_name)s"},{"line_number":77,"context_line":""},{"line_number":78,"context_line":"  The vGPU type to associate with the PCI devices has to be the section name"},{"line_number":79,"context_line":"  prefixed by ``vgpu_``. For example, for \u0027nvidia-11\u0027, you would declare"},{"line_number":80,"context_line":"  ``[vgpu_nvidia-11]/device_addresses``."},{"line_number":81,"context_line":""},{"line_number":82,"context_line":"  Each vGPU type also has to be declared in ``[devices]/enabled_vgpu_types``."},{"line_number":83,"context_line":""},{"line_number":84,"context_line":"  Related options:"},{"line_number":85,"context_line":""},{"line_number":86,"context_line":"  * ``[devices]/enabled_vgpu_types``"},{"line_number":87,"context_line":"  \"\"\"),"},{"line_number":88,"context_line":""},{"line_number":89,"context_line":"For example, it would be set in nova.conf:"},{"line_number":90,"context_line":""}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_2f64cb48","line":87,"range":{"start_line":61,"start_character":0,"end_line":87,"end_character":7},"in_reply_to":"5f7c97a3_f5324b0f","updated":"2018-06-07 21:29:15.000000000","message":"+1","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"0d9cc4757bd6bf95f26a29e37f80850ca191b4b9","unresolved":false,"context_lines":[{"line_number":91,"context_line":".. code::"},{"line_number":92,"context_line":""},{"line_number":93,"context_line":"  [devices]"},{"line_number":94,"context_line":"  enabled_vgpu_types \u003d nvidia-35,nvidia-36"},{"line_number":95,"context_line":"  [vgpu_nvidia-35]"},{"line_number":96,"context_line":"  device_addresses \u003d 0000:84:00.0,0000:85:00.0"},{"line_number":97,"context_line":"  [vgpu_nvidia-36]"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_6571cd77","line":94,"range":{"start_line":94,"start_character":23,"end_line":94,"end_character":42},"updated":"2018-06-05 13:39:33.000000000","message":"Where do there codenames come from? Could you provide a little info on this for those of us newer to vGPU stuff. This could be detailed in another spec so apologies if I missed that.","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6f7f4bc03e0818af11ae1b5ac2882f2edaa1977b","unresolved":false,"context_lines":[{"line_number":91,"context_line":".. code::"},{"line_number":92,"context_line":""},{"line_number":93,"context_line":"  [devices]"},{"line_number":94,"context_line":"  enabled_vgpu_types \u003d nvidia-35,nvidia-36"},{"line_number":95,"context_line":"  [vgpu_nvidia-35]"},{"line_number":96,"context_line":"  device_addresses \u003d 0000:84:00.0,0000:85:00.0"},{"line_number":97,"context_line":"  [vgpu_nvidia-36]"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_7035e9da","line":94,"range":{"start_line":94,"start_character":23,"end_line":94,"end_character":42},"in_reply_to":"5f7c97a3_6571cd77","updated":"2018-06-06 18:34:42.000000000","message":"Welcome to NVIDIA\u0027s \"standard\" mdev_supported_types.\n\nSee here:\n\nhttps://www.kernel.org/doc/Documentation/vfio-mediated-device.txt\n\n...\n\n* [mdev_supported_types]\n\n  The list of currently supported mediated device types and their details.\n\n  [\u003ctype-id\u003e], device_api, and available_instances are mandatory attributes\n  that should be provided by vendor driver.\n\nSo, to sum up, these type names come directly from the hard-coded product names that the vendor puts in their driver source code. \"nvidia-35\" is the vendor name \"nvidia\" appended to their internal product name, which is \"35\". Lovely, I know.\n\nSome background on the origins of VFIO mdev handling in libvirt:\n\nhttps://www.redhat.com/archives/libvir-list/2016-September/msg00750.html\n\nAlso, see this article from the oVirt authors describing the VFIO mdev stuff:\n\nhttps://mpolednik.github.io/2017/05/21/vfio-mdev/\n\nSpecifically, this part:\n\n\u003d\u003d begin quote \u003d\u003d\n\nWe can create and assign an instance of mdev type, but how do I figure out which types does my card support? Not easily yet. Let’s look at the information we can read from sysfs that may be relevant:\n\n $ tree -l -L 5 .\n ├── 0000:06:00.0 -\u003e ../../devices/pci0000:00/0000:00:03.0 /0000:04:00.0/0000:05:08.0/0000:06:00.0\n │   ├── mdev_supported_types\n │   │   ├── nvidia-11\n │   │   │   ├── description\n │   │   │   └── name\n\nShould be enough right? Except that description is literally free-form, vendor created text. No format, just anything.\n\n\u003d\u003d end quote \u003d\u003d","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"1ae0185636f30ef14e4e173af957de4c0068eaf1","unresolved":false,"context_lines":[{"line_number":91,"context_line":".. code::"},{"line_number":92,"context_line":""},{"line_number":93,"context_line":"  [devices]"},{"line_number":94,"context_line":"  enabled_vgpu_types \u003d nvidia-35,nvidia-36"},{"line_number":95,"context_line":"  [vgpu_nvidia-35]"},{"line_number":96,"context_line":"  device_addresses \u003d 0000:84:00.0,0000:85:00.0"},{"line_number":97,"context_line":"  [vgpu_nvidia-36]"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_ec4a5481","line":94,"range":{"start_line":94,"start_character":23,"end_line":94,"end_character":42},"in_reply_to":"5f7c97a3_7035e9da","updated":"2018-06-08 10:37:15.000000000","message":"Thanks for the context. Now I understand the Torvalds rant a little better :)","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"0d9cc4757bd6bf95f26a29e37f80850ca191b4b9","unresolved":false,"context_lines":[{"line_number":166,"context_line":"Other deployer impact"},{"line_number":167,"context_line":"---------------------"},{"line_number":168,"context_line":""},{"line_number":169,"context_line":"Operators need to either look at the sysfs (for libvirt) for knowing the"},{"line_number":170,"context_line":"existing pGPUs and which types are supported."},{"line_number":171,"context_line":""},{"line_number":172,"context_line":""},{"line_number":173,"context_line":"Developer impact"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_45e1312a","line":170,"range":{"start_line":169,"start_character":0,"end_line":170,"end_character":45},"updated":"2018-06-05 13:39:33.000000000","message":"You mean in order to configure nova.conf? I think this would be a problem for the deployment tooling more than the operator (unless they\u0027re writing these things by hand)","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"6f7f4bc03e0818af11ae1b5ac2882f2edaa1977b","unresolved":false,"context_lines":[{"line_number":166,"context_line":"Other deployer impact"},{"line_number":167,"context_line":"---------------------"},{"line_number":168,"context_line":""},{"line_number":169,"context_line":"Operators need to either look at the sysfs (for libvirt) for knowing the"},{"line_number":170,"context_line":"existing pGPUs and which types are supported."},{"line_number":171,"context_line":""},{"line_number":172,"context_line":""},{"line_number":173,"context_line":"Developer impact"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_b0b701f6","line":170,"range":{"start_line":169,"start_character":0,"end_line":170,"end_character":45},"in_reply_to":"5f7c97a3_45e1312a","updated":"2018-06-06 18:34:42.000000000","message":"Yes, it\u0027s a deployment tooling problem....","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"},{"author":{"_account_id":15334,"name":"Stephen Finucane","display_name":"stephenfin","email":"stephenfin@redhat.com","username":"sfinucan"},"change_message_id":"0d9cc4757bd6bf95f26a29e37f80850ca191b4b9","unresolved":false,"context_lines":[{"line_number":197,"context_line":"----------"},{"line_number":198,"context_line":""},{"line_number":199,"context_line":"* Create the config option"},{"line_number":200,"context_line":"* Modify the libvirt virt driver code to make use of that option for creating"},{"line_number":201,"context_line":"  the nested Resource Provider inventories."},{"line_number":202,"context_line":""},{"line_number":203,"context_line":"Dependencies"}],"source_content_type":"text/x-rst","patch_set":6,"id":"5f7c97a3_25bd1502","line":200,"range":{"start_line":200,"start_character":53,"end_line":200,"end_character":57},"updated":"2018-06-05 13:39:33.000000000","message":"the","commit_id":"a6cbe33c06bb265c90db7c3df68667e59dd95cc4"}]}