)]}'
{"specs/eventlet-to-something.rst":[{"author":{"_account_id":308,"name":"Thierry Carrez","email":"thierry@openstack.org","username":"ttx"},"change_message_id":"8592148b63ea51728d21855eb1bbf08791a6864a","unresolved":false,"context_lines":[{"line_number":313,"context_line":"already avoiding this kind of usage already (by using ``context`` variables"},{"line_number":314,"context_line":"for per-request state for example); certain things like ``oslo.config`` objects"},{"line_number":315,"context_line":"may need to be analyzed to determine thread-safety but most other usages"},{"line_number":316,"context_line":"should not be that problematic."},{"line_number":317,"context_line":""},{"line_number":318,"context_line":"Work Items"},{"line_number":319,"context_line":"----------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"9a80dd14_8e130db7","line":316,"updated":"2015-03-13 16:15:31.000000000","message":"Developers would also obviously be impacted by the quantity of code to review and change, as well as the time spent reviewing the resulting patchsets. Do we have an diea of how many lines of code are impacted ?","commit_id":"f049461704e4dda2f314e039a685d3299f23e78c"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"28fe36bccd9a235470cdbd620880d28f8d146052","unresolved":false,"context_lines":[{"line_number":313,"context_line":"already avoiding this kind of usage already (by using ``context`` variables"},{"line_number":314,"context_line":"for per-request state for example); certain things like ``oslo.config`` objects"},{"line_number":315,"context_line":"may need to be analyzed to determine thread-safety but most other usages"},{"line_number":316,"context_line":"should not be that problematic."},{"line_number":317,"context_line":""},{"line_number":318,"context_line":"Work Items"},{"line_number":319,"context_line":"----------"}],"source_content_type":"text/x-rst","patch_set":1,"id":"9a80dd14_7ad12f56","line":316,"in_reply_to":"9a80dd14_8e130db7","updated":"2015-03-13 17:15:23.000000000","message":"Nope, such quantification is not really possible without doing it IMHO.\nUnless u have other ideas on how to gather such quantification of LOC count...","commit_id":"f049461704e4dda2f314e039a685d3299f23e78c"},{"author":{"_account_id":308,"name":"Thierry Carrez","email":"thierry@openstack.org","username":"ttx"},"change_message_id":"8592148b63ea51728d21855eb1bbf08791a6864a","unresolved":false,"context_lines":[{"line_number":334,"context_line":"  executor..."},{"line_number":335,"context_line":"* Run tests..."},{"line_number":336,"context_line":"* See what fails (due to thread issues or races or other)..."},{"line_number":337,"context_line":"* Fix issues related to thread safety (or other) and repeat..."},{"line_number":338,"context_line":"* Figure out how to best inform operators about the new way this works and"},{"line_number":339,"context_line":"  any adjustements they would need to make."},{"line_number":340,"context_line":"* Profit!"}],"source_content_type":"text/x-rst","patch_set":1,"id":"9a80dd14_6eb89911","line":337,"updated":"2015-03-13 16:15:31.000000000","message":"That sounds like a great way to replace our old set of known issues with a fresh set of brand-new problems. How can we be reasonably sure we covered enough of those new race issues to not regress dramatically in quality ? By definition those kind of issues are hard to detect.","commit_id":"f049461704e4dda2f314e039a685d3299f23e78c"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"28fe36bccd9a235470cdbd620880d28f8d146052","unresolved":false,"context_lines":[{"line_number":334,"context_line":"  executor..."},{"line_number":335,"context_line":"* Run tests..."},{"line_number":336,"context_line":"* See what fails (due to thread issues or races or other)..."},{"line_number":337,"context_line":"* Fix issues related to thread safety (or other) and repeat..."},{"line_number":338,"context_line":"* Figure out how to best inform operators about the new way this works and"},{"line_number":339,"context_line":"  any adjustements they would need to make."},{"line_number":340,"context_line":"* Profit!"}],"source_content_type":"text/x-rst","patch_set":1,"id":"9a80dd14_9a3d0392","line":337,"in_reply_to":"9a80dd14_6eb89911","updated":"2015-03-13 17:15:23.000000000","message":"There is no magic bullet here, we need to understand there won\u0027t be.","commit_id":"f049461704e4dda2f314e039a685d3299f23e78c"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"6d2c9d76469b8f255f144b4eb1711c2e124369bf","unresolved":false,"context_lines":[{"line_number":32,"context_line":"#. Poor support for python 3.x (ongoing work related to this has been"},{"line_number":33,"context_line":"   slowly progressing for years). This limits the ability of OpenStack"},{"line_number":34,"context_line":"   projects to themselves move to python 3.x (as they depend on eventlet to"},{"line_number":35,"context_line":"   support it first)."},{"line_number":36,"context_line":"#. Incorrect, non-obvious monkeypatching order. It has required a watchful"},{"line_number":37,"context_line":"   eye and even a `best practices`_ specification of its own to try to"},{"line_number":38,"context_line":"   reconcile the different usages of eventlet (and the usages have varied"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_57325660","line":35,"updated":"2015-04-15 13:58:23.000000000","message":"I guess it\u0027s now not the case since py3 aware eventlet release was shipped.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"54546cf413bf552b4634e85156294721762f2eaf","unresolved":false,"context_lines":[{"line_number":32,"context_line":"#. Poor support for python 3.x (ongoing work related to this has been"},{"line_number":33,"context_line":"   slowly progressing for years). This limits the ability of OpenStack"},{"line_number":34,"context_line":"   projects to themselves move to python 3.x (as they depend on eventlet to"},{"line_number":35,"context_line":"   support it first)."},{"line_number":36,"context_line":"#. Incorrect, non-obvious monkeypatching order. It has required a watchful"},{"line_number":37,"context_line":"   eye and even a `best practices`_ specification of its own to try to"},{"line_number":38,"context_line":"   reconcile the different usages of eventlet (and the usages have varied"}],"source_content_type":"text/x-rst","patch_set":2,"id":"5aa745d1_2a257aaa","line":35,"in_reply_to":"5aa745d1_31650a45","updated":"2015-04-16 15:59:18.000000000","message":"Will do, thx victor :)","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9107,"name":"Victor Stinner","email":"vstinner@redhat.com","username":"haypo"},"change_message_id":"89335ee678a63d904ea814feb08122484610df2a","unresolved":false,"context_lines":[{"line_number":32,"context_line":"#. Poor support for python 3.x (ongoing work related to this has been"},{"line_number":33,"context_line":"   slowly progressing for years). This limits the ability of OpenStack"},{"line_number":34,"context_line":"   projects to themselves move to python 3.x (as they depend on eventlet to"},{"line_number":35,"context_line":"   support it first)."},{"line_number":36,"context_line":"#. Incorrect, non-obvious monkeypatching order. It has required a watchful"},{"line_number":37,"context_line":"   eye and even a `best practices`_ specification of its own to try to"},{"line_number":38,"context_line":"   reconcile the different usages of eventlet (and the usages have varied"}],"source_content_type":"text/x-rst","patch_set":2,"id":"5aa745d1_31650a45","line":35,"in_reply_to":"5aa745d1_89acbcee","updated":"2015-04-16 08:08:29.000000000","message":"Joshua, try eventlet 0.17.3, it should now work on Python 3.3 and 3.4 with monkey-patching.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"0c40181318faa61a50a9bf662dae0a44ff159104","unresolved":false,"context_lines":[{"line_number":32,"context_line":"#. Poor support for python 3.x (ongoing work related to this has been"},{"line_number":33,"context_line":"   slowly progressing for years). This limits the ability of OpenStack"},{"line_number":34,"context_line":"   projects to themselves move to python 3.x (as they depend on eventlet to"},{"line_number":35,"context_line":"   support it first)."},{"line_number":36,"context_line":"#. Incorrect, non-obvious monkeypatching order. It has required a watchful"},{"line_number":37,"context_line":"   eye and even a `best practices`_ specification of its own to try to"},{"line_number":38,"context_line":"   reconcile the different usages of eventlet (and the usages have varied"}],"source_content_type":"text/x-rst","patch_set":2,"id":"5aa745d1_89acbcee","line":35,"in_reply_to":"7aaa499b_57325660","updated":"2015-04-15 23:56:40.000000000","message":"My vote is out, for all I know the compat is still poor ;)","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"6d2c9d76469b8f255f144b4eb1711c2e124369bf","unresolved":false,"context_lines":[{"line_number":40,"context_line":"   and achieving consistency here has been a tax on developers as well as a"},{"line_number":41,"context_line":"   constant source of frustration and/or misunderstanding. These types of"},{"line_number":42,"context_line":"   issues can lead to lock-ups and/or races if done incorrectly..."},{"line_number":43,"context_line":"#. Fragility, it iss easy to forget to patch a function or have issues when"},{"line_number":44,"context_line":"   the standard library is modified. The eventlet port to Python 3 showed how"},{"line_number":45,"context_line":"   the patcher highly depends on the standard library. A recent eventlet"},{"line_number":46,"context_line":"   change (v0.16) \"turns off __builtin__ monkey patching by default\" to fix a"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_d73e6627","line":43,"updated":"2015-04-15 13:58:23.000000000","message":"iss -\u003e is","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2622,"name":"Samuel Merritt","email":"spam+launchpad@andcheese.org","username":"torgomatic"},"change_message_id":"3ecce6fa3bc30d986af9884f651d0608a6446000","unresolved":false,"context_lines":[{"line_number":48,"context_line":"   \u003chttps://bugs.launchpad.net/oslo.messaging/+bug/1369999\u003e`_ and `Second"},{"line_number":49,"context_line":"   simultaneous read on fileno can be raised on a closed socket #94"},{"line_number":50,"context_line":"   \u003chttps://github.com/eventlet/eventlet/issues/94\u003e`_ issues. Modules"},{"line_number":51,"context_line":"   implemented in C cannot be fully monkey-patched. Recent example: the `Fix"},{"line_number":52,"context_line":"   threading.Condition with monkey-patching on Python 3.3 and newer #187"},{"line_number":53,"context_line":"   \u003chttps://github.com/eventlet/eventlet/pull/187\u003e`_ change forces to use the"},{"line_number":54,"context_line":"   Python implementation of ``threading.RLock``, because the C implementation"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a41bdd9_6543eb13","line":51,"updated":"2015-07-15 21:39:01.000000000","message":"This depends on the module. C modules that do their own low-level socket IO will cause trouble, but C modules that call Python methods on socket objects can work just fine with eventlet (IIRC), as those method calls will end up finding the monkeypatched methods and doing the right thing.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2622,"name":"Samuel Merritt","email":"spam+launchpad@andcheese.org","username":"torgomatic"},"change_message_id":"3ecce6fa3bc30d986af9884f651d0608a6446000","unresolved":false,"context_lines":[{"line_number":65,"context_line":"#. `Action at a distance`_, lock-ups (and other issues/races) can happen at a"},{"line_number":66,"context_line":"   distance (in some library applications use, or a library of a library that"},{"line_number":67,"context_line":"   is being used...); so it is not always obvious where the problem actually"},{"line_number":68,"context_line":"   originates (this makes debugging much harder)."},{"line_number":69,"context_line":"#. Inability to move to standard and well accepted/well known how to operate"},{"line_number":70,"context_line":"   `wsgi`_ containers such as apache (via `mod_wsgi`_ or other)"},{"line_number":71,"context_line":"   nginx (or many others). `Keystone under apache`_ has been able to resolve"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a41bdd9_e5661bba","line":68,"updated":"2015-07-15 21:39:01.000000000","message":"This is hardly exclusive to eventlet.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"6d2c9d76469b8f255f144b4eb1711c2e124369bf","unresolved":false,"context_lines":[{"line_number":77,"context_line":"   inability to use standard containers make it that much harder for customers"},{"line_number":78,"context_line":"   (and especially established enterprise customers) to deploy openstack in"},{"line_number":79,"context_line":"   there environments (as they can not use standard containers and must"},{"line_number":80,"context_line":"   evaulate how they will deploy the openstack services under a new"},{"line_number":81,"context_line":"   non-standard container that likely has none of the existing functionality"},{"line_number":82,"context_line":"   or domain expertise that enterprise customers are dependent on and/or"},{"line_number":83,"context_line":"   require). This does **not** make operators lives easier and does not"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_17860edb","line":80,"updated":"2015-04-15 13:58:23.000000000","message":"evaulate -\u003e evaluate","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2622,"name":"Samuel Merritt","email":"spam+launchpad@andcheese.org","username":"torgomatic"},"change_message_id":"3ecce6fa3bc30d986af9884f651d0608a6446000","unresolved":false,"context_lines":[{"line_number":95,"context_line":"   http protocol/s). These communities typically ensure that support for new"},{"line_number":96,"context_line":"   http versions (`http 2.0`_ anyone?) and security bugs are fixed in a"},{"line_number":97,"context_line":"   reasonable amount of time. I am not sure the same can be said about"},{"line_number":98,"context_line":"   the in-built eventlet wsgi (which is really an add-on to eventlet, instead"},{"line_number":99,"context_line":"   of being the main application, like in apache, nginx) code that most"},{"line_number":100,"context_line":"   applications in openstack are required to use (due to the strong tie"},{"line_number":101,"context_line":"   to eventlet)."}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a41bdd9_626f61fc","line":98,"updated":"2015-07-15 21:39:01.000000000","message":"\"Add-on\" versus \"main application\" is a red herring. If we\u0027re going to consider time-to-security-fix and time-to-new-feature as reasons for dropping eventlet, let\u0027s do so based on actual historical data instead of throwing around pejoratives.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":179,"context_line":""},{"line_number":180,"context_line":"In general I would prefer **not** to use the above listed strategies and"},{"line_number":181,"context_line":"instead contribute to those libraries, making them better, and helping be good"},{"line_number":182,"context_line":"citizens in the larger python community."},{"line_number":183,"context_line":""},{"line_number":184,"context_line":"Forking"},{"line_number":185,"context_line":"*******"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_f0dd1242","line":182,"updated":"2015-03-24 15:04:08.000000000","message":"Do we know of any libraries we are currently using that are not thread-safe?","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9107,"name":"Victor Stinner","email":"vstinner@redhat.com","username":"haypo"},"change_message_id":"dffa6b6c5fb261a21cd3c6ee82d3cfbc09be0ef1","unresolved":false,"context_lines":[{"line_number":179,"context_line":""},{"line_number":180,"context_line":"In general I would prefer **not** to use the above listed strategies and"},{"line_number":181,"context_line":"instead contribute to those libraries, making them better, and helping be good"},{"line_number":182,"context_line":"citizens in the larger python community."},{"line_number":183,"context_line":""},{"line_number":184,"context_line":"Forking"},{"line_number":185,"context_line":"*******"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa963961_9f262c15","line":182,"in_reply_to":"9a80dd14_f0dd1242","updated":"2015-03-26 16:55:53.000000000","message":"\u003e Do we know of any libraries we are currently using that are not thread-safe?\n\nI don\u0027t know well third party modules, but I can say that they are modules in the Python standard library which are *not* thread safe. For example, the subprocess module is not thread-safe, especially in Python \u003c 3.2.\nhttp://bugs.python.org/issue19809\n\nI wrote a list of modules which are not thread-safe somewhere in the asyncio or the threads spec, but I\u0027m unable to find the comment :-(\n\nOther recent examples from the Python bug tracker.\n\n- zipfile is no thread-safe in python \u003c 3.5: http://bugs.python.org/issue14099\n- fnmatch is not thread-safe in python \u003c 3.5: http://bugs.python.org/issue23191\n\nThere is also the generic issue with C libraries calling directly fork() and the problem of inherited file descriptors with multiple threads. This issue is now fixed in Python 3.4:\nhttps://www.python.org/dev/peps/pep-0446/\n(I recently fixed a bug in nova, it came from libguestfs which calls fork() to run qemu in a subprocess. it was already fixed in more recent version of libguestfs which closes all file descriptors after fork, which is the right fix.)\n\nMany bugs have been fixed in the threading module, even recently. Last example, there was a race condition in the function generating an unique name for threads: http://bugs.python.org/issue11866. Other threads bugs:\n\n- threads.RLock doesn\u0027t handle signals correctly in Python \u003c 3.2: http://bugs.python.org/issue13697\n\nIn my experience, bugs related to threads are the most complex :-( For example, it took me 6 months with the help of colleagues to discover this annoying bug:\nhttp://bugs.python.org/issue14432\n(It\u0027s now fixed in Python 2.7.7)\n\nWell, as usually, the latest stable Python version has less bugs :-) So using threads in Python 3.4 is safer than using threads in Python 2.7.\n\nI\u0027m not trying that it\u0027s impossible to write safe code with threads in Python. I just want to warn you that there are many traps and that you must be more/very careful to avoid them :-)\n\nIf you don\u0027t run your whole application in threads, but only small functions, it\u0027s easier to check if the code executed in threads is thread-safe. If I understand correctly this spec, a large part of OpenStack will run concurrently in threads. So code requiring to be thread-safe become almost all OpenStack code base and all third-party modules, right?","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"d16756ffbe3c5bcb79c0c8f0ec6e44fa6850857a","unresolved":false,"context_lines":[{"line_number":179,"context_line":""},{"line_number":180,"context_line":"In general I would prefer **not** to use the above listed strategies and"},{"line_number":181,"context_line":"instead contribute to those libraries, making them better, and helping be good"},{"line_number":182,"context_line":"citizens in the larger python community."},{"line_number":183,"context_line":""},{"line_number":184,"context_line":"Forking"},{"line_number":185,"context_line":"*******"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a6ced46_dff42cd1","line":182,"in_reply_to":"9a80dd14_f0dd1242","updated":"2015-03-24 16:03:08.000000000","message":"I can\u0027t think of any off the top of my head; so that\u0027s a good sign, I guess the concern then really revolves around the projects thread safety; since I am hoping most of the libraries we are using are doing an \u0027ok\u0027 job at this already (its not really that hard as long as the library author/s think about it a little).","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":197,"context_line":"them; this likely *should* be the default for green-threads anyway as"},{"line_number":198,"context_line":"forcefully terminating a thread is typically bad practice, especially when"},{"line_number":199,"context_line":"the code being terminated is unable to have any response and/or become aware"},{"line_number":200,"context_line":"that it has been terminated forcefully)."},{"line_number":201,"context_line":""},{"line_number":202,"context_line":"WSGI"},{"line_number":203,"context_line":"~~~~"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_f0ab5285","line":200,"updated":"2015-03-24 15:04:08.000000000","message":"Is there any real difference to an application with \"daemon\" threads exiting completely over killing the green threads before exiting?","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9107,"name":"Victor Stinner","email":"vstinner@redhat.com","username":"haypo"},"change_message_id":"dffa6b6c5fb261a21cd3c6ee82d3cfbc09be0ef1","unresolved":false,"context_lines":[{"line_number":197,"context_line":"them; this likely *should* be the default for green-threads anyway as"},{"line_number":198,"context_line":"forcefully terminating a thread is typically bad practice, especially when"},{"line_number":199,"context_line":"the code being terminated is unable to have any response and/or become aware"},{"line_number":200,"context_line":"that it has been terminated forcefully)."},{"line_number":201,"context_line":""},{"line_number":202,"context_line":"WSGI"},{"line_number":203,"context_line":"~~~~"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa963961_df8274c8","line":200,"in_reply_to":"9a80dd14_f0ab5285","updated":"2015-03-26 16:55:53.000000000","message":"Daemon threads cause various crashes at Python shutdown, please avoid them.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"d16756ffbe3c5bcb79c0c8f0ec6e44fa6850857a","unresolved":false,"context_lines":[{"line_number":197,"context_line":"them; this likely *should* be the default for green-threads anyway as"},{"line_number":198,"context_line":"forcefully terminating a thread is typically bad practice, especially when"},{"line_number":199,"context_line":"the code being terminated is unable to have any response and/or become aware"},{"line_number":200,"context_line":"that it has been terminated forcefully)."},{"line_number":201,"context_line":""},{"line_number":202,"context_line":"WSGI"},{"line_number":203,"context_line":"~~~~"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a6ced46_ff9f285d","line":200,"in_reply_to":"9a80dd14_f0ab5285","updated":"2015-03-24 16:03:08.000000000","message":"Not really I guess, both are killing threads without there consent; which can leave things in funky states.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":212,"context_line":"deployment time a deployer wants to use apache (or wants to use something"},{"line_number":213,"context_line":"else) or wants to shove the wsgi entrypoint into a *hacky* (and typically"},{"line_number":214,"context_line":"non-standard/questionably solid IMHO) eventlet wsgi server that is up to there"},{"line_number":215,"context_line":"own deployment decisions/choices."},{"line_number":216,"context_line":""},{"line_number":217,"context_line":"For those wsgi applications that interact with the previously mentioned message"},{"line_number":218,"context_line":"queue based services it should be fine for each WSGI request to use a blocking"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_90905644","line":215,"updated":"2015-03-24 15:04:08.000000000","message":"So you mean that the WSGI container should manage the threads, and our WSGI applications should be written to be thread-safe, but not try to manage separate request-processing threads directly?","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9107,"name":"Victor Stinner","email":"vstinner@redhat.com","username":"haypo"},"change_message_id":"dffa6b6c5fb261a21cd3c6ee82d3cfbc09be0ef1","unresolved":false,"context_lines":[{"line_number":212,"context_line":"deployment time a deployer wants to use apache (or wants to use something"},{"line_number":213,"context_line":"else) or wants to shove the wsgi entrypoint into a *hacky* (and typically"},{"line_number":214,"context_line":"non-standard/questionably solid IMHO) eventlet wsgi server that is up to there"},{"line_number":215,"context_line":"own deployment decisions/choices."},{"line_number":216,"context_line":""},{"line_number":217,"context_line":"For those wsgi applications that interact with the previously mentioned message"},{"line_number":218,"context_line":"queue based services it should be fine for each WSGI request to use a blocking"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa963961_a241d33b","line":215,"in_reply_to":"1a6ced46_3f89c026","updated":"2015-03-26 16:55:53.000000000","message":"Agreed.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":12363,"name":"Oleksii Chuprykov","email":"a.chuprikoff@gmail.com","username":"ochuprykov"},"change_message_id":"47d739555f8e3aad8327d39e18df24ec97f7bf2c","unresolved":false,"context_lines":[{"line_number":212,"context_line":"deployment time a deployer wants to use apache (or wants to use something"},{"line_number":213,"context_line":"else) or wants to shove the wsgi entrypoint into a *hacky* (and typically"},{"line_number":214,"context_line":"non-standard/questionably solid IMHO) eventlet wsgi server that is up to there"},{"line_number":215,"context_line":"own deployment decisions/choices."},{"line_number":216,"context_line":""},{"line_number":217,"context_line":"For those wsgi applications that interact with the previously mentioned message"},{"line_number":218,"context_line":"queue based services it should be fine for each WSGI request to use a blocking"}],"source_content_type":"text/x-rst","patch_set":2,"id":"da9b358b_91e0ec97","line":215,"in_reply_to":"1a6ced46_3f89c026","updated":"2015-04-01 11:21:30.000000000","message":"Can we simply use apache mpm prefork for applications that are (perhaps) non thread-safe or it is hard to make them such?\nSeems, keystone run in such mode now.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"d16756ffbe3c5bcb79c0c8f0ec6e44fa6850857a","unresolved":false,"context_lines":[{"line_number":212,"context_line":"deployment time a deployer wants to use apache (or wants to use something"},{"line_number":213,"context_line":"else) or wants to shove the wsgi entrypoint into a *hacky* (and typically"},{"line_number":214,"context_line":"non-standard/questionably solid IMHO) eventlet wsgi server that is up to there"},{"line_number":215,"context_line":"own deployment decisions/choices."},{"line_number":216,"context_line":""},{"line_number":217,"context_line":"For those wsgi applications that interact with the previously mentioned message"},{"line_number":218,"context_line":"queue based services it should be fine for each WSGI request to use a blocking"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a6ced46_3f89c026","line":215,"in_reply_to":"9a80dd14_90905644","updated":"2015-03-24 16:03:08.000000000","message":"I think so, ya.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"ad838bd8e1182b49ad593cf6b65531a754172464","unresolved":false,"context_lines":[{"line_number":212,"context_line":"deployment time a deployer wants to use apache (or wants to use something"},{"line_number":213,"context_line":"else) or wants to shove the wsgi entrypoint into a *hacky* (and typically"},{"line_number":214,"context_line":"non-standard/questionably solid IMHO) eventlet wsgi server that is up to there"},{"line_number":215,"context_line":"own deployment decisions/choices."},{"line_number":216,"context_line":""},{"line_number":217,"context_line":"For those wsgi applications that interact with the previously mentioned message"},{"line_number":218,"context_line":"queue based services it should be fine for each WSGI request to use a blocking"}],"source_content_type":"text/x-rst","patch_set":2,"id":"baa041b7_ecb5907e","line":215,"in_reply_to":"da9b358b_91e0ec97","updated":"2015-04-01 19:39:33.000000000","message":"It\u0027s not hard; the issue imho is that not all projects would still be using apache mpm (just the wsgi ones can be); so since we share common code/libraries between the different components even if we avoid thread-safety in the wsgi parts; we\u0027d have to think about it later (or also say that we will not have threads elsewhere also, which is a thing we could say...)\n\nFor example:\n\n  - nova-api (move to wsgi)\n  - cinder-api (move to wsgi)\n  - glance-api/registry (move to wsgi)\n  - nova-compute (what to do with it?)\n  - nova-scheduler (what to do with it?)\n\nThe following is sorta interesting (although old...)\n\nhttp://www.radwin.org/michael/talks/yapache-oscon2006.pdf\n\n^ about apache + mpm @ yahoo (from 2006.... so dated...)","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"6d2c9d76469b8f255f144b4eb1711c2e124369bf","unresolved":false,"context_lines":[{"line_number":218,"context_line":"queue based services it should be fine for each WSGI request to use a blocking"},{"line_number":219,"context_line":"executor (oslo.messaging provides one of these) or a thread-pool executor that"},{"line_number":220,"context_line":"the WSGI server maintains (or keep a pool of executors alive so that"},{"line_number":221,"context_line":"each WSGI request that comes in can grap a executor from the pool and use it"},{"line_number":222,"context_line":"as needed, and release that executor back to the pool when done)."},{"line_number":223,"context_line":""},{"line_number":224,"context_line":"Periodics"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_62890e75","line":221,"updated":"2015-04-15 13:58:23.000000000","message":"grap -\u003e grab?","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":219,"context_line":"executor (oslo.messaging provides one of these) or a thread-pool executor that"},{"line_number":220,"context_line":"the WSGI server maintains (or keep a pool of executors alive so that"},{"line_number":221,"context_line":"each WSGI request that comes in can grap a executor from the pool and use it"},{"line_number":222,"context_line":"as needed, and release that executor back to the pool when done)."},{"line_number":223,"context_line":""},{"line_number":224,"context_line":"Periodics"},{"line_number":225,"context_line":"~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_f0e7d2e0","line":222,"updated":"2015-03-24 15:04:08.000000000","message":"If we use the threaded executor, would we have to worry about the WSGI container shutting down the process containing daemonized threads before those threads are done sending a message?","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"d16756ffbe3c5bcb79c0c8f0ec6e44fa6850857a","unresolved":false,"context_lines":[{"line_number":219,"context_line":"executor (oslo.messaging provides one of these) or a thread-pool executor that"},{"line_number":220,"context_line":"the WSGI server maintains (or keep a pool of executors alive so that"},{"line_number":221,"context_line":"each WSGI request that comes in can grap a executor from the pool and use it"},{"line_number":222,"context_line":"as needed, and release that executor back to the pool when done)."},{"line_number":223,"context_line":""},{"line_number":224,"context_line":"Periodics"},{"line_number":225,"context_line":"~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a6ced46_ff76483d","line":222,"in_reply_to":"9a80dd14_f0e7d2e0","updated":"2015-03-24 16:03:08.000000000","message":"Good question, will look into it.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":243,"context_line":"It will be expected that these types of eventlet specific usage will"},{"line_number":244,"context_line":"need to be explored in more depth and analyzed to determine exactly the best"},{"line_number":245,"context_line":"way to convert them from what they are *today* to there equivalent (or nearly"},{"line_number":246,"context_line":"equivalent) future version."},{"line_number":247,"context_line":""},{"line_number":248,"context_line":"Locks locks and more locks"},{"line_number":249,"context_line":"~~~~~~~~~~~~~~~~~~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_30224a01","line":246,"updated":"2015-03-24 15:04:08.000000000","message":"A separate flow will work for some periodic tasks. Some might need to modify state within the application (I\u0027m not sure, just conjecturing), and those would need to stay in the process to have access to whatever state is being managed. Either way, we have a couple of approaches to replace the existing implementation and it ought to be possible to do it without requiring massive application changes unless that\u0027s the way we want to do it.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"6d2c9d76469b8f255f144b4eb1711c2e124369bf","unresolved":false,"context_lines":[{"line_number":261,"context_line":"due to how any races around I/O like function would have already been"},{"line_number":262,"context_line":"uncovered)."},{"line_number":263,"context_line":""},{"line_number":264,"context_line":"The amount of code we will hit that has threading issues (and doesn\u0027t use"},{"line_number":265,"context_line":"locks) is a definite unknown (but does seem solveable and tractable). It may"},{"line_number":266,"context_line":"be useful to do a analysis of patterns that projects are using and to have"},{"line_number":267,"context_line":"each projects PTL provide a summary/basic analysis of there feeling for there"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_82bc1add","line":264,"updated":"2015-04-15 13:58:23.000000000","message":"The last time I checked, http://git.openstack.org/cgit/openstack/neutron/tree/neutron/agent/l3/router_processing_queue.py was relying on the fact that eventlet does not yield during non I/O operations.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"0c40181318faa61a50a9bf662dae0a44ff159104","unresolved":false,"context_lines":[{"line_number":261,"context_line":"due to how any races around I/O like function would have already been"},{"line_number":262,"context_line":"uncovered)."},{"line_number":263,"context_line":""},{"line_number":264,"context_line":"The amount of code we will hit that has threading issues (and doesn\u0027t use"},{"line_number":265,"context_line":"locks) is a definite unknown (but does seem solveable and tractable). It may"},{"line_number":266,"context_line":"be useful to do a analysis of patterns that projects are using and to have"},{"line_number":267,"context_line":"each projects PTL provide a summary/basic analysis of there feeling for there"}],"source_content_type":"text/x-rst","patch_set":2,"id":"5aa745d1_89939ca3","line":264,"in_reply_to":"7aaa499b_82bc1add","updated":"2015-04-15 23:56:40.000000000","message":"Interesting, sounds badly coupled/to tightly coupled imho, but thanks for the example.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":268,"context_line":"projects thread-safety (it would be nice to do a similar analysis of any"},{"line_number":269,"context_line":"libraries that are being used). This kind of proactive (instead of reactive)"},{"line_number":270,"context_line":"analysis would help in determining issues ahead of time, instead of when those"},{"line_number":271,"context_line":"libraries or projects start causing hard to detect bugs/errors/issues."},{"line_number":272,"context_line":""},{"line_number":273,"context_line":"Caches"},{"line_number":274,"context_line":"******"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_50741efd","line":271,"updated":"2015-03-24 15:04:08.000000000","message":"It would be useful to do some research, but it would also be useful to start creating some feature branches to work on moving applications over. We don\u0027t have to shift all of OpenStack, or even all of a given project, at one time.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"6d2c9d76469b8f255f144b4eb1711c2e124369bf","unresolved":false,"context_lines":[{"line_number":268,"context_line":"projects thread-safety (it would be nice to do a similar analysis of any"},{"line_number":269,"context_line":"libraries that are being used). This kind of proactive (instead of reactive)"},{"line_number":270,"context_line":"analysis would help in determining issues ahead of time, instead of when those"},{"line_number":271,"context_line":"libraries or projects start causing hard to detect bugs/errors/issues."},{"line_number":272,"context_line":""},{"line_number":273,"context_line":"Caches"},{"line_number":274,"context_line":"******"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_a26716b6","line":271,"in_reply_to":"1a6ced46_9feb94af","updated":"2015-04-15 13:58:23.000000000","message":"I don\u0027t think feature branches are applicable in openstack community. Neutron had bad experience with those.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"d16756ffbe3c5bcb79c0c8f0ec6e44fa6850857a","unresolved":false,"context_lines":[{"line_number":268,"context_line":"projects thread-safety (it would be nice to do a similar analysis of any"},{"line_number":269,"context_line":"libraries that are being used). This kind of proactive (instead of reactive)"},{"line_number":270,"context_line":"analysis would help in determining issues ahead of time, instead of when those"},{"line_number":271,"context_line":"libraries or projects start causing hard to detect bugs/errors/issues."},{"line_number":272,"context_line":""},{"line_number":273,"context_line":"Caches"},{"line_number":274,"context_line":"******"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a6ced46_9feb94af","line":271,"in_reply_to":"9a80dd14_50741efd","updated":"2015-03-24 16:03:08.000000000","message":"Agreed, it\u0027s probably really not that hard either to do some hacks to see what this does/how it initially goes, most of the pieces are there I think (or could be put in place in a semi-working way relatively easily I think)...","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"6d2c9d76469b8f255f144b4eb1711c2e124369bf","unresolved":false,"context_lines":[{"line_number":289,"context_line":"external state that is being maintained (in databases, using lock files,"},{"line_number":290,"context_line":"libvirt databases ...). These kind of state (and the issues associated with"},{"line_number":291,"context_line":"them) are an already known and addressed (or are being addressed) issue since"},{"line_number":292,"context_line":"there usage is independent of green-threads (or native-threads). The problems"},{"line_number":293,"context_line":"that exist with these types of external state (and coordination/usage) exist no"},{"line_number":294,"context_line":"matter what internal application solution (greenthreads, asyncio, threads...)"},{"line_number":295,"context_line":"is finally determined to be the path forward."}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_027fea15","line":292,"updated":"2015-04-15 13:58:23.000000000","message":"there -\u003e their","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":315,"context_line":"reproducible by having a *idle* thread that is dedicated to doing this (if its"},{"line_number":316,"context_line":"existence is still justified now that things like ``gdb`` will operate more"},{"line_number":317,"context_line":"naturally with regular threads and without the funkiness that required having a"},{"line_number":318,"context_line":"`backdoor`_ in the first place)."},{"line_number":319,"context_line":""},{"line_number":320,"context_line":"REST API impact"},{"line_number":321,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_909696ee","line":318,"updated":"2015-03-24 15:04:08.000000000","message":"We should probably default to reproducing the feature somehow.\n\nWe also have the reports library work that makes it easy to dump information from a process for offline review.\n\nIt would be useful to know if operators use the backdoor at all, or if that\u0027s just used in development environments.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"e59933126f503cc728f69bf77e3fcd6a3f416923","unresolved":false,"context_lines":[{"line_number":315,"context_line":"reproducible by having a *idle* thread that is dedicated to doing this (if its"},{"line_number":316,"context_line":"existence is still justified now that things like ``gdb`` will operate more"},{"line_number":317,"context_line":"naturally with regular threads and without the funkiness that required having a"},{"line_number":318,"context_line":"`backdoor`_ in the first place)."},{"line_number":319,"context_line":""},{"line_number":320,"context_line":"REST API impact"},{"line_number":321,"context_line":"---------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"da9b358b_bf58ae57","line":318,"in_reply_to":"9a80dd14_909696ee","updated":"2015-03-30 15:24:42.000000000","message":"Agreed; let me post an operator(s) mail to there mailing list. I\u0027d also like to know if people actually use the backdoor (maybe they don\u0027t even know about it...)","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":12363,"name":"Oleksii Chuprykov","email":"a.chuprikoff@gmail.com","username":"ochuprykov"},"change_message_id":"47d739555f8e3aad8327d39e18df24ec97f7bf2c","unresolved":false,"context_lines":[{"line_number":318,"context_line":"`backdoor`_ in the first place)."},{"line_number":319,"context_line":""},{"line_number":320,"context_line":"REST API impact"},{"line_number":321,"context_line":"---------------"},{"line_number":322,"context_line":""},{"line_number":323,"context_line":"Depending on the above mentioned WSGI section and how it would be"},{"line_number":324,"context_line":"advantegous to remove eventlet/(green)thread concepts entirely from those"}],"source_content_type":"text/x-rst","patch_set":2,"id":"da9b358b_94e0da97","line":321,"updated":"2015-04-01 11:21:30.000000000","message":"Another thing to discuss. There is a proposal to use standard 80/443 port for all http web-services. Such change can be done using apache named virtual hosts and require some changes in cmd clients/ keystone service catalog\n(another parts of the code where we send requests)","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":324,"context_line":"advantegous to remove eventlet/(green)thread concepts entirely from those"},{"line_number":325,"context_line":"layers this would have an impact on the operators running those API(s) as"},{"line_number":326,"context_line":"they would have to make a decision about which WSGI hosting container (apache,"},{"line_number":327,"context_line":"nginx, uWSGI...) to use."},{"line_number":328,"context_line":""},{"line_number":329,"context_line":"Performance Impact"},{"line_number":330,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_d0b3ae97","line":327,"updated":"2015-03-24 15:04:08.000000000","message":"We should pick one as a recommendation. Apache seems perfectly fine as a default, since that\u0027s what the keystone team is already using. If we find issues, we can change. That said, picking a default WSGI container can be a separate spec if we need it.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"e59933126f503cc728f69bf77e3fcd6a3f416923","unresolved":false,"context_lines":[{"line_number":324,"context_line":"advantegous to remove eventlet/(green)thread concepts entirely from those"},{"line_number":325,"context_line":"layers this would have an impact on the operators running those API(s) as"},{"line_number":326,"context_line":"they would have to make a decision about which WSGI hosting container (apache,"},{"line_number":327,"context_line":"nginx, uWSGI...) to use."},{"line_number":328,"context_line":""},{"line_number":329,"context_line":"Performance Impact"},{"line_number":330,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"da9b358b_7f18e684","line":327,"in_reply_to":"9a80dd14_d0b3ae97","updated":"2015-03-30 15:24:42.000000000","message":"I\u0027m fine with that also; apache + yahoo has been used for over 10+ years and it was worked out just fine... Maybe I can get someone at yahoo to write up how it has worked on a blog or something, to ensure that people understand that a large tech company has used apache for years (without many issues)....","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":12363,"name":"Oleksii Chuprykov","email":"a.chuprikoff@gmail.com","username":"ochuprykov"},"change_message_id":"47d739555f8e3aad8327d39e18df24ec97f7bf2c","unresolved":false,"context_lines":[{"line_number":324,"context_line":"advantegous to remove eventlet/(green)thread concepts entirely from those"},{"line_number":325,"context_line":"layers this would have an impact on the operators running those API(s) as"},{"line_number":326,"context_line":"they would have to make a decision about which WSGI hosting container (apache,"},{"line_number":327,"context_line":"nginx, uWSGI...) to use."},{"line_number":328,"context_line":""},{"line_number":329,"context_line":"Performance Impact"},{"line_number":330,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"da9b358b_d44ae225","line":327,"in_reply_to":"da9b358b_7f18e684","updated":"2015-04-01 11:21:30.000000000","message":"It seems, that we don\u0027t have to make additional changes for using apache (besides those in api clients). So using apache by default is reasonable.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"0c40181318faa61a50a9bf662dae0a44ff159104","unresolved":false,"context_lines":[{"line_number":324,"context_line":"advantegous to remove eventlet/(green)thread concepts entirely from those"},{"line_number":325,"context_line":"layers this would have an impact on the operators running those API(s) as"},{"line_number":326,"context_line":"they would have to make a decision about which WSGI hosting container (apache,"},{"line_number":327,"context_line":"nginx, uWSGI...) to use."},{"line_number":328,"context_line":""},{"line_number":329,"context_line":"Performance Impact"},{"line_number":330,"context_line":"------------------"}],"source_content_type":"text/x-rst","patch_set":2,"id":"5aa745d1_69e73041","line":327,"in_reply_to":"da9b358b_d44ae225","updated":"2015-04-15 23:56:40.000000000","message":"I\u0027d like that to :)","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"6d2c9d76469b8f255f144b4eb1711c2e124369bf","unresolved":false,"context_lines":[{"line_number":334,"context_line":"to be and what impact it will have on request rate (if any). For this"},{"line_number":335,"context_line":"we will need to determine how many green-threads a single regular thread"},{"line_number":336,"context_line":"can perform the equivalent work of (perhaps the ratio is 1:1, although it"},{"line_number":337,"context_line":"is likley not). It should be relatively simple to increase the ratio as we"},{"line_number":338,"context_line":"see fit and address bottlenecks that we find (as long as those bottlenecks"},{"line_number":339,"context_line":"are not python itself, which they might actually be...)."},{"line_number":340,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_42fcd271","line":337,"updated":"2015-04-15 13:58:23.000000000","message":"likley -\u003e likely","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"6d2c9d76469b8f255f144b4eb1711c2e124369bf","unresolved":false,"context_lines":[{"line_number":340,"context_line":""},{"line_number":341,"context_line":"For certain *compute* (and not I/O) intensive projects (keystone?) they"},{"line_number":342,"context_line":"may not see much advantage of switching to native threads (due to the GIL)"},{"line_number":343,"context_line":"and they may have to try alternate techniques to solve there architectural"},{"line_number":344,"context_line":"limitations (scaling out with processes...). This is **not** expected to be"},{"line_number":345,"context_line":"new territory for these projects as they *should have* hit the same kind of"},{"line_number":346,"context_line":"issues already with eventlet at scale (if they didn\u0027t that project seems"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_a2edd6b8","line":343,"updated":"2015-04-15 13:58:23.000000000","message":"there -\u003e their","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":344,"context_line":"limitations (scaling out with processes...). This is **not** expected to be"},{"line_number":345,"context_line":"new territory for these projects as they *should have* hit the same kind of"},{"line_number":346,"context_line":"issues already with eventlet at scale (if they didn\u0027t that project seems"},{"line_number":347,"context_line":"somewhat questionable)."},{"line_number":348,"context_line":""},{"line_number":349,"context_line":"Things that would likely work better"},{"line_number":350,"context_line":"~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_c409332c","line":347,"updated":"2015-03-24 15:04:08.000000000","message":"I believe that\u0027s what drove keystone to drop eventlet in favor of regular threading and Apache integration. I hope some of the keystone team will provide more details about their experiences there.\n\nThe multiprocessing library API is meant to be extremely similar to the threading API so that it is easy to change between threads and processes, but have them managed in a similar way. We may see some benefit to doing that, especially on multi-core servers.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9107,"name":"Victor Stinner","email":"vstinner@redhat.com","username":"haypo"},"change_message_id":"dffa6b6c5fb261a21cd3c6ee82d3cfbc09be0ef1","unresolved":false,"context_lines":[{"line_number":344,"context_line":"limitations (scaling out with processes...). This is **not** expected to be"},{"line_number":345,"context_line":"new territory for these projects as they *should have* hit the same kind of"},{"line_number":346,"context_line":"issues already with eventlet at scale (if they didn\u0027t that project seems"},{"line_number":347,"context_line":"somewhat questionable)."},{"line_number":348,"context_line":""},{"line_number":349,"context_line":"Things that would likely work better"},{"line_number":350,"context_line":"~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa963961_62550bc5","line":347,"in_reply_to":"9a80dd14_c409332c","updated":"2015-03-26 16:55:53.000000000","message":"\u003e The multiprocessing library API is meant to be extremely similar to the threading API so that it is easy to change between threads and processes,\n\nUse concurrent.futures instead: you can choose between threads or processes, it\u0027s the same API. When using processes, data are serialized, which may have an impact on performances. So you have to benchmark to choose the executor. At least, you don\u0027t have to modify the code.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"e59933126f503cc728f69bf77e3fcd6a3f416923","unresolved":false,"context_lines":[{"line_number":344,"context_line":"limitations (scaling out with processes...). This is **not** expected to be"},{"line_number":345,"context_line":"new territory for these projects as they *should have* hit the same kind of"},{"line_number":346,"context_line":"issues already with eventlet at scale (if they didn\u0027t that project seems"},{"line_number":347,"context_line":"somewhat questionable)."},{"line_number":348,"context_line":""},{"line_number":349,"context_line":"Things that would likely work better"},{"line_number":350,"context_line":"~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"}],"source_content_type":"text/x-rst","patch_set":2,"id":"da9b358b_3a25ccaa","line":347,"in_reply_to":"fa963961_62550bc5","updated":"2015-03-30 15:24:42.000000000","message":"Agreed, use the futures interface. Although in taskflow (where both can be used) the process pool one in older python versions is pretty busted (or has quite a few bugs); ones I found I put @ http://docs.openstack.org/developer/taskflow/engines.html#parallel (the red box there).","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":360,"context_line":"~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"},{"line_number":361,"context_line":""},{"line_number":362,"context_line":"* Hashlib in python 2.6 apparently does not release the GIL (this is fixed"},{"line_number":363,"context_line":"  in `4751`_ and is no longer a problem in python 2.7 or greater)."},{"line_number":364,"context_line":"* All those libraries that forgot to think about threads (and are racey and/or"},{"line_number":365,"context_line":"  broken due to this)."},{"line_number":366,"context_line":""}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_4443833e","line":363,"updated":"2015-03-24 15:04:08.000000000","message":"We have already dropped 2.6 support in the servers, and will be dropping it in the libraries, soon. When we have stable branches of the libraries we can probably drop support from master as long as we keep it in the stable branches.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":371,"context_line":""},{"line_number":372,"context_line":"Developers will now have to think more about what may block and where that"},{"line_number":373,"context_line":"code that is blocking is running (if it\u0027s running in one of these per-request"},{"line_number":374,"context_line":"threads that most OpenStack applications use, then they likely don\u0027t need to"},{"line_number":375,"context_line":"care). They may need to think about locks on shared data (especially anything"},{"line_number":376,"context_line":"regarding global or shared object data/state); although most projects are"},{"line_number":377,"context_line":"already avoiding this kind of usage already (by using ``context`` variables"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_c47cd373","line":374,"updated":"2015-03-24 15:04:08.000000000","message":"Do they, though? I/O will still cause the GIL to be released, and so too will more system calls.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"e59933126f503cc728f69bf77e3fcd6a3f416923","unresolved":false,"context_lines":[{"line_number":371,"context_line":""},{"line_number":372,"context_line":"Developers will now have to think more about what may block and where that"},{"line_number":373,"context_line":"code that is blocking is running (if it\u0027s running in one of these per-request"},{"line_number":374,"context_line":"threads that most OpenStack applications use, then they likely don\u0027t need to"},{"line_number":375,"context_line":"care). They may need to think about locks on shared data (especially anything"},{"line_number":376,"context_line":"regarding global or shared object data/state); although most projects are"},{"line_number":377,"context_line":"already avoiding this kind of usage already (by using ``context`` variables"}],"source_content_type":"text/x-rst","patch_set":2,"id":"da9b358b_da34a0df","line":374,"in_reply_to":"9a80dd14_c47cd373","updated":"2015-03-30 15:24:42.000000000","message":"Fair question.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":401,"context_line":"* Fix issues related to thread safety (or other) and repeat..."},{"line_number":402,"context_line":"* Figure out how to best inform operators about the new way this works and"},{"line_number":403,"context_line":"  any adjustements they would need to make."},{"line_number":404,"context_line":"* Profit!"},{"line_number":405,"context_line":""},{"line_number":406,"context_line":"Asyncio proposal"},{"line_number":407,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_e4cab74d","line":404,"updated":"2015-03-24 15:04:08.000000000","message":"We should decide what criteria is good enough to consider it safe to formally move a project. Some period of time without gate races, maybe? One successful test run is probably  not enough.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":1297,"name":"Joshua Harlow","email":"jxharlow@godaddy.com","username":"harlowja"},"change_message_id":"e59933126f503cc728f69bf77e3fcd6a3f416923","unresolved":false,"context_lines":[{"line_number":401,"context_line":"* Fix issues related to thread safety (or other) and repeat..."},{"line_number":402,"context_line":"* Figure out how to best inform operators about the new way this works and"},{"line_number":403,"context_line":"  any adjustements they would need to make."},{"line_number":404,"context_line":"* Profit!"},{"line_number":405,"context_line":""},{"line_number":406,"context_line":"Asyncio proposal"},{"line_number":407,"context_line":"\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d"}],"source_content_type":"text/x-rst","patch_set":2,"id":"da9b358b_9a4df874","line":404,"in_reply_to":"9a80dd14_e4cab74d","updated":"2015-03-30 15:24:42.000000000","message":"Agreed","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"6d2c9d76469b8f255f144b4eb1711c2e124369bf","unresolved":false,"context_lines":[{"line_number":409,"context_line":"Proposed change"},{"line_number":410,"context_line":"---------------"},{"line_number":411,"context_line":""},{"line_number":412,"context_line":"This specification proposes to replace eventlet with asyncio. It reduces the"},{"line_number":413,"context_line":"risk of race conditions, fix eventlet issues, prepare OpenStack for the future"},{"line_number":414,"context_line":"(reduce the technical debt) and ease the usage of native threads (for better"},{"line_number":415,"context_line":"performances)."}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_0205aa44","line":412,"updated":"2015-04-15 13:58:23.000000000","message":"Really? I thought the whole spec intended to promote native threads.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":412,"context_line":"This specification proposes to replace eventlet with asyncio. It reduces the"},{"line_number":413,"context_line":"risk of race conditions, fix eventlet issues, prepare OpenStack for the future"},{"line_number":414,"context_line":"(reduce the technical debt) and ease the usage of native threads (for better"},{"line_number":415,"context_line":"performances)."},{"line_number":416,"context_line":""},{"line_number":417,"context_line":"Even if the title contains \"asyncio\", the spec proposes to use trollius. The"},{"line_number":418,"context_line":"name asyncio is used in the spec because it is better known than trollius, and"}],"source_content_type":"text/x-rst","patch_set":2,"id":"9a80dd14_c4fe9369","line":415,"updated":"2015-03-24 15:04:08.000000000","message":"Do we have any sort of evidence that OpenStack\u0027s overall communication patterns benefit from asyncio vs. threads?","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9107,"name":"Victor Stinner","email":"vstinner@redhat.com","username":"haypo"},"change_message_id":"dffa6b6c5fb261a21cd3c6ee82d3cfbc09be0ef1","unresolved":false,"context_lines":[{"line_number":412,"context_line":"This specification proposes to replace eventlet with asyncio. It reduces the"},{"line_number":413,"context_line":"risk of race conditions, fix eventlet issues, prepare OpenStack for the future"},{"line_number":414,"context_line":"(reduce the technical debt) and ease the usage of native threads (for better"},{"line_number":415,"context_line":"performances)."},{"line_number":416,"context_line":""},{"line_number":417,"context_line":"Even if the title contains \"asyncio\", the spec proposes to use trollius. The"},{"line_number":418,"context_line":"name asyncio is used in the spec because it is better known than trollius, and"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa963961_c2e17700","line":415,"in_reply_to":"9a80dd14_c4fe9369","updated":"2015-03-26 16:55:53.000000000","message":"I don\u0027t understand your question. Which kind of benefit? Performances?","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":2472,"name":"Doug Hellmann","email":"dhellmann@redhat.com","username":"doug-hellmann"},"change_message_id":"a53c599170254a7691bcfa5f92d8d9e5b1bdc017","unresolved":false,"context_lines":[{"line_number":420,"context_line":""},{"line_number":421,"context_line":"The spec doesn\u0027t change OpenStack components running WSGI servers like"},{"line_number":422,"context_line":"nova-api. The specific problem of using asyncio with WSGI will need a separated"},{"line_number":423,"context_line":"spec."},{"line_number":424,"context_line":""},{"line_number":425,"context_line":"The spec is focused on Oslo Messaging and Ceilometer projects, except of the"},{"line_number":426,"context_line":"``ceilometer-api`` daemon. More OpenStack applications may be modified later if"}],"source_content_type":"text/x-rst","patch_set":2,"id":"1a6ced46_26e38134","line":423,"updated":"2015-03-24 15:04:08.000000000","message":"I don\u0027t think we can safely ignore the fact that a lot of our code runs in a WSGI server and talks to a database. A solution that doesn\u0027t fully take those needs into account isn\u0027t going to be able to be used widely, which means we will have to support multiple patterns, which means libraries and application code will be more complicated to implement and support. If we decide that we want to use asyncio, we need to have a plan for the database and WSGI use cases before we start updating a lot of application code.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9107,"name":"Victor Stinner","email":"vstinner@redhat.com","username":"haypo"},"change_message_id":"dffa6b6c5fb261a21cd3c6ee82d3cfbc09be0ef1","unresolved":false,"context_lines":[{"line_number":420,"context_line":""},{"line_number":421,"context_line":"The spec doesn\u0027t change OpenStack components running WSGI servers like"},{"line_number":422,"context_line":"nova-api. The specific problem of using asyncio with WSGI will need a separated"},{"line_number":423,"context_line":"spec."},{"line_number":424,"context_line":""},{"line_number":425,"context_line":"The spec is focused on Oslo Messaging and Ceilometer projects, except of the"},{"line_number":426,"context_line":"``ceilometer-api`` daemon. More OpenStack applications may be modified later if"}],"source_content_type":"text/x-rst","patch_set":2,"id":"fa963961_222b83ed","line":423,"in_reply_to":"1a6ced46_26e38134","updated":"2015-03-26 16:55:53.000000000","message":"I\u0027m not working currently on asyncio. I know that I have to update my spec (or this spec) to elaborate my plan for WSGI. It\u0027s what I suggested on the mailing list and what Joshua just explain above. For WSGI, just write blocking call and let the HTTP server decide how to parallelize the code: threads and/or processes. \"Just\" make sure that your code is thread-safe.\n\nLater we can investigate the option of a fully asynchronous server asyncio. We may get better performances:\nhttp://blog.gmludo.eu/2015/03/benchmark-python-web-production-stack.html\n\nBut I don\u0027t think that the WSGI part is the most interesting right now. Writing blocking code and let the server parallelize is probably the best compromise today to get rid of eventlet.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"},{"author":{"_account_id":9656,"name":"Ihar Hrachyshka","email":"ihrachys@redhat.com","username":"ihrachys","status":"Red Hat Networking Systems Engineer"},"change_message_id":"6d2c9d76469b8f255f144b4eb1711c2e124369bf","unresolved":false,"context_lines":[{"line_number":446,"context_line":"Assignee(s)"},{"line_number":447,"context_line":"-----------"},{"line_number":448,"context_line":""},{"line_number":449,"context_line":"Assignee is for moving these guidelines through the review process to"},{"line_number":450,"context_line":"something that we all agree on. The expectation is that these become"},{"line_number":451,"context_line":"review criteria that we can reference and are implemented by a large"},{"line_number":452,"context_line":"number of people than listed below (one person will **not** be able to"}],"source_content_type":"text/x-rst","patch_set":2,"id":"7aaa499b_629c6e0f","line":449,"updated":"2015-04-15 13:58:23.000000000","message":"The example section can be removed.","commit_id":"d7e5a573702c3dc1f122ea342ad71a8bacdd3078"}]}