)]}'
{"specs/newton/approved/image-prop-pci-mappings.rst":[{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"04f33ca912465b30cf87ac1739513f399533628c","unresolved":false,"context_lines":[{"line_number":24,"context_line":"on how to identify the tagged device in guest. This approach requires the"},{"line_number":25,"context_line":"applications running in VM to be modified in order to understand information"},{"line_number":26,"context_line":"passed in metadata. None of applications available today are able to work with"},{"line_number":27,"context_line":"this new interface. Existing applications often require assignment of"},{"line_number":28,"context_line":"application specific PCI addresses to network interfaces present in the guest."},{"line_number":29,"context_line":"Assigning specific PCI addresses to network interfaces in the guest is not"},{"line_number":30,"context_line":"supported by Nova, hence deployment of such applications with Nova is not"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_ffe60baf","line":27,"updated":"2016-05-23 21:08:48.000000000","message":"Would you elaborate a bit more here about what existing applications require assignment of specific PCI addresses to NICs in the guest? Is there a particular application or VNF codebase that you have in mind? How does this application *currently* run in a virtualized environment if it depends on hard-coded PCI addresses?","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":12126,"name":"Gerardo García de Blas","email":"gerardo.garciadeblas@gmail.com","username":"ggdb"},"change_message_id":"46403cb060cce6d3196589c0f189cb20b51cf2de","unresolved":false,"context_lines":[{"line_number":24,"context_line":"on how to identify the tagged device in guest. This approach requires the"},{"line_number":25,"context_line":"applications running in VM to be modified in order to understand information"},{"line_number":26,"context_line":"passed in metadata. None of applications available today are able to work with"},{"line_number":27,"context_line":"this new interface. Existing applications often require assignment of"},{"line_number":28,"context_line":"application specific PCI addresses to network interfaces present in the guest."},{"line_number":29,"context_line":"Assigning specific PCI addresses to network interfaces in the guest is not"},{"line_number":30,"context_line":"supported by Nova, hence deployment of such applications with Nova is not"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_13b0ca42","line":27,"in_reply_to":"bab6814e_573a49ce","updated":"2016-05-25 12:13:01.000000000","message":"The point is not whether virt-device-role-tagging is a good solution or not.\n\nThe point is whether we can use this mechanism coupled with the addition of image metadata containing a mapping of tags to virtual PCI addresses.\nglance image-update nfv_vnf_image \\\n   --property hw_pci_mappings \u003d \\\n   \"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\"\n\nThe virt drivers would be extended to match tags from image metadata to tags from instance and configure the devices pci addresses accordingly. The initial implementation will be done for the Libvirt driver.\n\nIt seems that this is not perceived as the \"canonical solution\", but the proposed solution is undeterministic waiting for applications/cloud-init to understand the instance metadata and configure interfaces accordingly. The result is that, until this is not fixed, Openstack is not usable for serious NFV use cases.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":21626,"name":"Ricardo","email":"rnoriega@redhat.com","username":"rnoriega"},"change_message_id":"e1b85598d02748de6d431d25797bfa617c35bc4f","unresolved":false,"context_lines":[{"line_number":24,"context_line":"on how to identify the tagged device in guest. This approach requires the"},{"line_number":25,"context_line":"applications running in VM to be modified in order to understand information"},{"line_number":26,"context_line":"passed in metadata. None of applications available today are able to work with"},{"line_number":27,"context_line":"this new interface. Existing applications often require assignment of"},{"line_number":28,"context_line":"application specific PCI addresses to network interfaces present in the guest."},{"line_number":29,"context_line":"Assigning specific PCI addresses to network interfaces in the guest is not"},{"line_number":30,"context_line":"supported by Nova, hence deployment of such applications with Nova is not"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_573a49ce","line":27,"in_reply_to":"bab6814e_8135baf8","updated":"2016-05-25 09:08:04.000000000","message":"I agree that libvirt allows to assign specific PCI addresses, but in a controlled manner over the virtual infrastructure. OpenStack can manage other hypervisors as well, like XEN, HyperV, etc. which could emulate different chipsets, PCI vs PCIe... so Virtual Device Role Tagging is a generic solution from Nova\u0027s point of view.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":19285,"name":"Alfonso Tierno","email":"alf.tierno@gmail.com"},"change_message_id":"7c2b343364bfbd3b707da4fb9ee1f22473a84626","unresolved":false,"context_lines":[{"line_number":24,"context_line":"on how to identify the tagged device in guest. This approach requires the"},{"line_number":25,"context_line":"applications running in VM to be modified in order to understand information"},{"line_number":26,"context_line":"passed in metadata. None of applications available today are able to work with"},{"line_number":27,"context_line":"this new interface. Existing applications often require assignment of"},{"line_number":28,"context_line":"application specific PCI addresses to network interfaces present in the guest."},{"line_number":29,"context_line":"Assigning specific PCI addresses to network interfaces in the guest is not"},{"line_number":30,"context_line":"supported by Nova, hence deployment of such applications with Nova is not"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_277cde1f","line":27,"in_reply_to":"bab6814e_8135baf8","updated":"2016-05-24 15:27:24.000000000","message":"The virtual PCI address is needed for identifying the interfaces when you have more than one and you want to know e.g. which one is connected to \"A\". The MAC address is not a good solution in a virtualized environment (same VM can be instantiate twice).\nIn cloud environment there is not such a problem because interfaces get IP address by DHCP on all its interfaces and this IP address is used to identify which one is connected to e.g. \"A\". But this is not the case in NFV where manual IP can be set inside the VNF instead of DHCP (using OSS for example); or you can use non IP protocols for transport.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":19212,"name":"Antonio López","email":"antonio.lopezgracia@telefonica.com","username":"alogra"},"change_message_id":"b06a9e3b370499f5c95caf5f0684aa172ddf4378","unresolved":false,"context_lines":[{"line_number":24,"context_line":"on how to identify the tagged device in guest. This approach requires the"},{"line_number":25,"context_line":"applications running in VM to be modified in order to understand information"},{"line_number":26,"context_line":"passed in metadata. None of applications available today are able to work with"},{"line_number":27,"context_line":"this new interface. Existing applications often require assignment of"},{"line_number":28,"context_line":"application specific PCI addresses to network interfaces present in the guest."},{"line_number":29,"context_line":"Assigning specific PCI addresses to network interfaces in the guest is not"},{"line_number":30,"context_line":"supported by Nova, hence deployment of such applications with Nova is not"}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_8135baf8","line":27,"in_reply_to":"bab6814e_ffe60baf","updated":"2016-05-24 15:08:12.000000000","message":"DPDK based VNFs use PCI addresses to identify interfaces. Libvirt allows to assign specific PCI addresses to virtual NICs, that\u0027s how industrial VNFs are currently being tested in Telefonica NFV Reference Lab.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"04f33ca912465b30cf87ac1739513f399533628c","unresolved":false,"context_lines":[{"line_number":39,"context_line":"to appear at specific virtual PCI addresses. For example the application"},{"line_number":40,"context_line":"expects the management interface to appear at PCI address 00:02.0 and two"},{"line_number":41,"context_line":"dataplane interfaces, dataplane1 and dataplane2,to appear at 00:03.0"},{"line_number":42,"context_line":"and 00:04.0."},{"line_number":43,"context_line":""},{"line_number":44,"context_line":"The tenant needs to be able to specify applications requirements in image"},{"line_number":45,"context_line":"metadata and identify the network interfaces using virtual device role tagging."}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_dfeb4777","line":42,"updated":"2016-05-23 21:08:48.000000000","message":"Instead of changing the virtualization infrastructure to specify hard-coded PCI addresses for virtual NICs, wouldn\u0027t it be easier and more flexible to simply change the tenant application to not depend on hard-coded PCI addresses and instead either query a metadata service (ala the virt-device-role-tagging spec) or select a particular NIC based on a query of the NIC\u0027s capabilities?","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":11647,"name":"Przemyslaw Czesnowicz","email":"przemyslaw.czesnowicz@intel.com","username":"pczesno"},"change_message_id":"23d1613b24955df86b36c187302ee00b02193ea0","unresolved":false,"context_lines":[{"line_number":39,"context_line":"to appear at specific virtual PCI addresses. For example the application"},{"line_number":40,"context_line":"expects the management interface to appear at PCI address 00:02.0 and two"},{"line_number":41,"context_line":"dataplane interfaces, dataplane1 and dataplane2,to appear at 00:03.0"},{"line_number":42,"context_line":"and 00:04.0."},{"line_number":43,"context_line":""},{"line_number":44,"context_line":"The tenant needs to be able to specify applications requirements in image"},{"line_number":45,"context_line":"metadata and identify the network interfaces using virtual device role tagging."}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_61a792bd","line":42,"in_reply_to":"bab6814e_84782891","updated":"2016-05-26 08:06:33.000000000","message":"From what I\u0027ve been told there are many applications that need this kind of functionality, so the easier solution is to modify Nova.I mean \"Easier\" in terms of how much work would have to be done, how many projects would have to be modified, certified etc., doesn\u0027t include convincing Nova community.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":19212,"name":"Antonio López","email":"antonio.lopezgracia@telefonica.com","username":"alogra"},"change_message_id":"b06a9e3b370499f5c95caf5f0684aa172ddf4378","unresolved":false,"context_lines":[{"line_number":39,"context_line":"to appear at specific virtual PCI addresses. For example the application"},{"line_number":40,"context_line":"expects the management interface to appear at PCI address 00:02.0 and two"},{"line_number":41,"context_line":"dataplane interfaces, dataplane1 and dataplane2,to appear at 00:03.0"},{"line_number":42,"context_line":"and 00:04.0."},{"line_number":43,"context_line":""},{"line_number":44,"context_line":"The tenant needs to be able to specify applications requirements in image"},{"line_number":45,"context_line":"metadata and identify the network interfaces using virtual device role tagging."}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_84782891","line":42,"in_reply_to":"bab6814e_dfeb4777","updated":"2016-05-24 15:08:12.000000000","message":"That will be the case long term. But we would like to enable Openstack to deploy VNFs in the short term too.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"fe8192ea9c1781def9e9d59c9f612a460a30b5b0","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a122d0e_9faa0019","line":72,"updated":"2016-05-04 19:00:58.000000000","message":"This is allowing the tenant user / cloud admin to explicitly assign guest PCI addresses. This is something we explicitly *DO NOT* want to ever allow in Nova.\n\nThe whole point of the role device tagging concept is that the user tags devices on the host and Nova expposes metadata to the guest associating these tags with the addresses assigned by the hypervisor.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":7,"name":"Jay Pipes","email":"jaypipes@gmail.com","username":"jaypipes"},"change_message_id":"04f33ca912465b30cf87ac1739513f399533628c","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_3f01b33a","line":72,"updated":"2016-05-23 21:08:48.000000000","message":"Yeah, I agree with the Dans here. This is proposing to leak way too much application-implementation information into the calling interface between the Compute API and the image service. The above is really the domain of hardware-specific application code not virtualized or cloud use cases...","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":19212,"name":"Antonio López","email":"antonio.lopezgracia@telefonica.com","username":"alogra"},"change_message_id":"537919c7590d267c4785dd4070b39d77550f4422","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_a52f54f5","line":72,"in_reply_to":"1a122d0e_9c730516","updated":"2016-05-18 06:55:49.000000000","message":"Sorry, I don\u0027t understand. Why is this removing the abstraction? We are allowing to specify the vCPU topology, I do not see the differences with this case, that is allowing to specify virtual pci addresses.\nThis spec is specifying characteristics of the guest, required at least for VNFs. If any hypervisor is not allowing to set the virtual pci addresses then in that case it won\u0027t be allowed. But what is the problem with that? Or openstack is only specifying functions that are possible to implement in all hypervisors?","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":4393,"name":"Dan Smith","email":"dms@danplanet.com","username":"danms"},"change_message_id":"8773eee654eb3d7844d62d21f14457b5dc060f65","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"1a122d0e_9c730516","line":72,"in_reply_to":"1a122d0e_9faa0019","updated":"2016-05-05 14:31:27.000000000","message":"Completely agree. Exposing this level of detail outside of nova removes much of the abstraction we should be providing, IMHO. I\u0027m also quite sure you can specify things this way that won\u0027t be supportable by some hypervisors, and/or clash with default devices.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":19212,"name":"Antonio López","email":"antonio.lopezgracia@telefonica.com","username":"alogra"},"change_message_id":"537919c7590d267c4785dd4070b39d77550f4422","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_659c4c6a","line":72,"in_reply_to":"1a122d0e_9faa0019","updated":"2016-05-18 06:55:49.000000000","message":"Why this is something you do not want to ever allow in Nova? What is the reasoning for that? I would like to understand it. \nThis spec is just specifying virtual pci addresses for the guest, is unrelated to the host or hypervisor. I see it similar to the -allowed- specification of the vCPU topology.\nThis spec is very important to be able to deploy safely VNFs in Openstack without the need to reengineer them to autoconfigure from metadata.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"8a6415174bd22d9634916f9a265cdc73bad93182","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"7aa08908_a9760604","line":72,"in_reply_to":"7aa08908_4c3bfb38","updated":"2016-06-09 13:24:31.000000000","message":"Different machine chipsets can have completely different PCI bus layouts. With the change from PIIX4 to Q35 in particular the layout will look very different, because Q35 will be primarily a PCI-E based chipset, while PIIX4 is plain PCI. With PCI-E, Q35 can have multiple PCI switches, to which PCI-E compatible devices are attached, as well as bridge to provide support for legacy PCI devices. The PCI address given to a device determines which  bridge/bus/switch it is attached to. So if you just used the same PCI address that you have previously used for PIIX4 chipset, it is likely that the devices will end up attached to an incompatible, or at least sub-optimal bus/switch. \n\nFurther, currently there is just a single PCI root for guests, even if they have multiple NUMA nodes. This is bad because the guest OS cannot determine which node devices have memory affinity with. QEMU has added the ability to have multiple PCI roots, each associated with particular guest NUMA nodes. So when we have a guest that spans multiple NUMA nodes, we will be able to associate host PCI devices witrh specific  guest NUMA nodes, so that guest OS can optimize usage of the device to avoid cross-numa node memory access. This kind of setup again requires Nova to be in charge of assigning PCI addresses as the tenant user has no idea what NUMA node each assigned device must be associated with.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":12126,"name":"Gerardo García de Blas","email":"gerardo.garciadeblas@gmail.com","username":"ggdb"},"change_message_id":"f076f726e6d2c8268c26d0bbadbd588d4c627adf","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"7aa08908_459e5e08","line":72,"in_reply_to":"7aa08908_a9760604","updated":"2016-06-09 15:03:39.000000000","message":"Regarding what Daniel has commented: \"So if you just used the same PCI address that you have previously used for PIIX4 chipset, it is likely that the devices will end up attached to an incompatible, or at least sub-optimal bus/switch.\"\n\nWhen you say sub-optimal, I understand that it is related to PCI signalling but not to data transfering when using PMD. I can confirm that we can obtain full line rate with 64 byte frames using those PCI addresses in the so old PIIX4.\n\nRegarding the comment \"currently there is just a sinle PCI root for guests, even if they have multiple NUMA nodes. This is bad because the guest OS cannot determine which node devices have memory affinity with.\"\n\n99% of network applications use a single NUMA node, so this is not a real problem for the moment.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":19212,"name":"Antonio López","email":"antonio.lopezgracia@telefonica.com","username":"alogra"},"change_message_id":"1ffa6b186103a63c59fe53d963309dfb2611bfcd","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"7aa08908_4c3bfb38","line":72,"in_reply_to":"9abb7d3a_8eb47382","updated":"2016-06-06 16:55:20.000000000","message":"Can you please elaborate on why we cannot expect those pci addresses to still be appropriate when changing to the Q35 chipset?","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"0f34b77950e70e0bba048b13c5f9895f653dde1f","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_38e4cd39","line":72,"in_reply_to":"bab6814e_0d28ade1","updated":"2016-05-19 14:03:26.000000000","message":"BTW, I understand that there is a desire to be able to use pre-existing guest OS which don\u0027t know about the nova device metadata concept without having to modify the guest OS image. From Nova POV though the ability to run pre-existing guest OS designed for a non-cloud system is not a general goal - if it works then great, but its by luck rather than by design. The intention is that guest OS images be explicitly built with cloud deployment in mind. That all said, we are aiming to minimize the work involved in adapting pre-existing OS to work under nova. This is why we intend to provide some level of tool support to integrate with cloud-init \u0026 udev, so that applications can run a simple query at startup to ask udev what nic/disk has the tag \"foo\". ie we don\u0027t want every app to have to reinvent the logic for parsing \u0026 interpreting the nova JSON metadata about device tags - we want to solve that problem for all, so they can just do a single udev query (or similar) to get the device info.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":19285,"name":"Alfonso Tierno","email":"alf.tierno@gmail.com"},"change_message_id":"ab57e892ae73acd764b6bb97a47ebf5f8b6d0b79","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_f7389d3f","line":72,"in_reply_to":"bab6814e_1c82f200","updated":"2016-05-19 11:46:18.000000000","message":"IMHO for me abstraction means that the Nova does not expose the physical hardware,topology, processor features, ... used for deploy a VM. However to allow VM with a concrete virtualized hardware, topology, devices, ... is something desirable, the most attributes that can be specified the better. This vPCI injection solves the problem of interface identification needed when there is no IP, or when linux kernel is not used for choosing the output interface based on the destination IP (for example when using DPDK). It is true that the proposed alternative of metadata tagging can be used, but it is also true that this alternative will force to change the current linux/cloud_init start scheduling","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":19212,"name":"Antonio López","email":"antonio.lopezgracia@telefonica.com","username":"alogra"},"change_message_id":"7659a3c62b364b04e89161011dc1295e9c4835f1","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_9f13180d","line":72,"in_reply_to":"bab6814e_38e4cd39","updated":"2016-05-19 16:06:47.000000000","message":"Thanks for the explanation, Daniel.\n\nI do not understand why \"users would just have to \"pick \u0026 pray\", trying addresses until they found one that happened to work\". You mean physical addresses, or virtual addresses? If it is the latter, why those cannot be arbitrary?","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":12126,"name":"Gerardo García de Blas","email":"gerardo.garciadeblas@gmail.com","username":"ggdb"},"change_message_id":"5e937e9265d96c2e75926fa2cab6ae66af06b8ee","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_7a8ce67f","line":72,"in_reply_to":"bab6814e_3f01b33a","updated":"2016-05-23 22:49:30.000000000","message":"Well, I tend to disagree on how things should progress in this industry, but I keep on learning the hard way. Unfortunately, perfect is the enemy of good, and this kind of decisions condition greatly the industrial usage of a SW until things become usable.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":12126,"name":"Gerardo García de Blas","email":"gerardo.garciadeblas@gmail.com","username":"ggdb"},"change_message_id":"460086294e754b50b111da3552451c88806a892b","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_cd586535","line":72,"in_reply_to":"bab6814e_8aef6386","updated":"2016-05-19 13:48:56.000000000","message":"Daniel, are we talking about the same? I have the impression that you are confusing here PCI addresses in the host with virtual PCI addresses in the guest. All this spec is about virtual PCI address, similarly to the vCPU topology.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"9307bb0c91f128fac5fb800c3c86958337ab30a3","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"9abb7d3a_8eb47382","line":72,"in_reply_to":"bab6814e_990fb1e0","updated":"2016-05-31 10:41:57.000000000","message":"Those addresses happen to work because you don\u0027t have any other devices present in those slots and we\u0027ve not changed the machine type (aka base chipset) on x86 for a good while. - it is still the severely ancient PIIX4 (from circa 1995). In the not too distant future QEMU \u0026 OpenStack are quite likely to change to use a Q35 chipset by default which has a radically different PCI bus layout than PIIX4, since it supports PCI-E, instead of plain PCI. When this happens you can\u0027t expect those PCI addresses you\u0027ve used to still be appropriate. When designing Nova APIs / features we need to consider what is likely to happen in the future, not merely what has happened to work in the past.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":12126,"name":"Gerardo García de Blas","email":"gerardo.garciadeblas@gmail.com","username":"ggdb"},"change_message_id":"8999a08b6116fe990815b560e352fcc38b866b05","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_990fb1e0","line":72,"in_reply_to":"bab6814e_9aafa693","updated":"2016-05-25 14:49:15.000000000","message":"By the way, I am a bit surprised after re-reading the sentence \"You can\u0027t pick an arbitrary virtual address and expect it to magically just work - you need an address that correctly corresponds with the bus the device will attach to\".\n\nIt has been 5 years now from the first deployment of a VNF-like virtual machine we made in our lab using directly libvirt (virsh define) from XML that we made ourselves, without relying at all in Openstack. During these years, we have been using the following virtual PCI addresses without problems: 0000:00:0[3-9a-f], 0000:00:1[0-6]. \n\nBased on my experience, the previous PCI addresses worked flawlessly in all cases during these 5 years, using different versions of RHEL, qemu and libvirt.\n\nCould you please let us know a counter-example using the previous PCI addresses? I would like to know if there are corner cases that we are omitting?","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":12126,"name":"Gerardo García de Blas","email":"gerardo.garciadeblas@gmail.com","username":"ggdb"},"change_message_id":"7c8a01b3d3933d063d62a381ab76f4003a8af47f","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_d7f072cb","line":72,"in_reply_to":"bab6814e_9aafa693","updated":"2016-05-23 12:30:13.000000000","message":"I don\u0027t get the point.\n\nIn the worst case, if the PCI address specified by the VNF vendor in the image cannot be used because qemu does not allow it, then the deployment will fail.\nBut if it succeeds, the advantage is clear from the point of view of the VNF vendor, whose current VNFs will work without requiring any update.\n\nThe ideal solution would be to modify cloud-init process to read the PCI information from meta-data. However, while this is not possible, I don\u0027t understand why this is not allowed as a transition mechanism.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"472461e66c51305d191fb005806cadf176b9e19e","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_9aafa693","line":72,"in_reply_to":"bab6814e_9f13180d","updated":"2016-05-19 16:23:11.000000000","message":"\"virtual\" does *NOT* imply \"arbitrary\".\n\nA QEMU virtual machine is emulating real hardware and thus there are constraints over what addresses you can use for devices and what buses you can plug what devices into. eg if QEMU provides a machine with both a PCI and PCI-E bus, the choice of virtual address for the device is related to which bus you plug the device into. You can\u0027t pick an arbitrary virtual address and expect it to magically just work - you need an address that correctly corresponds with the bus the device will attach to, and corresponds with a free slot in that bus. The traditional QEMU PIIX4 chipset is rather boring \u0026 simple to address for (as long as you have less than ~28 devices), but with the QEMU Q35 or ICH9 chipsets, the rules are very complex","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":12126,"name":"Gerardo García de Blas","email":"gerardo.garciadeblas@gmail.com","username":"ggdb"},"change_message_id":"9e6334852738ad5ad91d14c09c15886349d917d6","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_1c82f200","line":72,"in_reply_to":"bab6814e_a52f54f5","updated":"2016-05-18 09:50:24.000000000","message":"I totally agree that this is not different from the case of the specification of a vCPU topology.\nI am adding below 2 use cases for NFV where this specification of vPCI addresses makes sense:\n\n1. You have a lab environment to test deployments before moving to production environment. You deploy there a virtual machine using PF passthrough interfaces (that VM is prepared to deal with packets at high performance rates). Inside that VM, you configure IP addresses, firewall rules, etc. Now you want to move that VM from the lab environment to the production environment. When deployed on the production environment, during the booting process the VM will not be able to identify the interfaces properly and will assign names in an unpredictable way. As a result, the configuration that was done inside the VM will be totally incorrect.\n\n2. You have deployed in your production environment a virtual machine using PF passthrough interfaces (that VM is again a VM prepared to deal with packets at high performance rates). Inside that VM, you configure IP addresses, firewall rules, etc. Now you want to migrate that VM to a different availability zone, but you don\u0027t want to lose the configuration that was done inside that VM. In order to guarantee integrity, virtual HW in the destination host must be preserved during that migration.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"33fa859191643ba6e49bb5b16135b9db98d51e97","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_0d28ade1","line":72,"in_reply_to":"bab6814e_cd586535","updated":"2016-05-19 13:57:09.000000000","message":"No, I\u0027m not mixed up. I\u0027m explicitly. referring to guest PCI device addresses. It is nothing like as simple as vCPU topology.\n\nGiven a guest with 8 CPUs, you can easily request any combination of sockets/cores/threads without knowing anything else about the VM configuration or anything about the hypervisor behind it.\n\nWhen you start assigning PCI addresses there is a huge knowledge burden. First you need to know what machine type (piix4, q35, etc) is in use, which varies according to QEMU version, hardware architecture, hypervisor vendor, nova config options. You need to understand the hardware constraints of that specific machine type - eg what PCI buses / bridges / switches it exposes. Which of them are PCI vs PCI-e. Which slots already have devices present in them. Whether you\u0027ll need to be able to do hotplug of the devices. None of this information is made available to users of Nova, so if they did have ability to assign PCI addresses, users would just have to \"pick \u0026 pray\", trying addresses until they found one that happened to work. Even then there\u0027s zero guarantee it\u0027d work next time the VM boots, as the VM may run on a different hypervisor version where different slots are free in the guest VM. The user is always going to loose at this game since they are playing it blindfold with both arms tied behind their back.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"},{"author":{"_account_id":1779,"name":"Daniel Berrange","email":"berrange@redhat.com","username":"berrange"},"change_message_id":"ad9d2d025f08ff458a0f4f94efadc4a48fb74f29","unresolved":false,"context_lines":[{"line_number":69,"context_line":""},{"line_number":70,"context_line":"\tglance image-update nfv_vnf_image \\"},{"line_number":71,"context_line":"\t\t--property hw_pci_mappings \u003d \\"},{"line_number":72,"context_line":"\t\t\t\"mgmt_if\":\"00:02.0\",\"nfv1\":\"00:03.0\",\"nfv2\":\"00:04.0\""},{"line_number":73,"context_line":""},{"line_number":74,"context_line":"Boot a virtual machine using nfv_vnf image and tag the interfaces:"},{"line_number":75,"context_line":""}],"source_content_type":"text/x-rst","patch_set":1,"id":"bab6814e_8aef6386","line":72,"in_reply_to":"bab6814e_f7389d3f","updated":"2016-05-19 13:45:04.000000000","message":"The cloud user who is responsible for booting VMs has little to no apriori knowledge about the specifics of the virtual machine hardware configuration they are getting. They can make some general high level requests, such as asking for a disk or NIC but they don\u0027t have any visibility into the broader configuration of the guest - there are a multitude of other hardware devices and tunables that are set on the VM on their behalf by nova/libvirt/qemu.  There is no way for the cloud user to know, from the nova APIs, what PCI slots are suitable and/or available for use. This information varies depending on the version of QEMU \u0026 Libvirt deployed, the configuration options of nova \u0026 libvirt host, the type of flavour, the hardware architecture in use and more besides. So even if the user tried booting a machine, analysing it to try and reverse engineer this information, anything they learn can be invalidated at any moment because the cloud admin could upgrade their software, or the VM may simply be placed on a different node to the one they analysed. To make it practical to support device address assignment in the nova API, nova would have to expose massively more information about virtual machines upfront and this is not a direction that we\u0027re interested in going down. It turns nova back into a traditional data center virtualization application, throwing away the benefits of cloud.\n\nComparing this to specifying vCPU topology or NUMA topology is a false comparison. Neither of these have the kind of constraints on their usage I describe above \u0026 can be trivially specified without having to know anything about the VM hardware upfront.  What you\u0027re asking for is more akin to requesting that Nova let you specify individual virtual memory DIMMs in each NUMA node.","commit_id":"4d1420868f28721b0d48ee7789b3b20dbd3fc90a"}]}