aleix's blog

BitPacket: Python 2.x and 3.0 compatibility

14 October 2010 8:09 pm (bitpacket | python)

Lately, I've been porting BitPacket to Python 3.0. I wanted to keep backwards compatibility with Python 2.6 (which is the 2.x I have in my Debian) and, thankfully, I only had to fix three minor issues:

  • Unicode strings

  • Dictionary keys

  • Bytes vs. strings

StringIO and unicode strings

If you have ever used the StringIO module you should be familiar with this:

    from cStringIO import StringIO
except ImportError:
    from StringIO import StringIO

In Py3k the StringIO is located under the io package, so you should changed the above by:

from io import StringIO

which is also compatible with Python 2.6.

Once I did the change my code only worked in Py3k, Python 2.6 complained when trying to use the write method with a simple string:

>>> from io import StringIO
>>> stream = StringIO()
>>> stream.write("test")
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python2.6/", line 1515, in write
TypeError: can't write str to text stream

You should note that in Py3k all the strings are unicode strings by default. This is not true in Python 2.6, so my first approach was the following:

>>> stream.write(u"test")

Unfortunately, this only worked in Python 2.6. Py3k does not recognize the unicode prefix "u", giving you this error:

>>> stream.write(u"test")
  File "", line 1
SyntaxError: invalid syntax

I googled a bit and find out a Making code compatible with Python 2 and 3 post (from the guy that finished all the SICP exercices) where it explained some similar issues, so I came up with this solution:

def u_str(string):
    if sys.hexversion >= 0x03000000:
        return string
        return unicode(string)

>>> stream.write(u_str("test"))

In Py3k, unicode does not exist but as that line is never executed we don't get any error.

Even that worked well, I was not very happy with it. It was too slow and I had to use the custom u_str function everywhere. So, I googled a bit more and I found a nice pycon 2009 talk about Python 3.0 compatibility. Finally, I had which I think is the best solution (for both speed and clearness):

    # This will raise an exception in Py3k, as unicode doesn't exist
    str = unicode

So, instead of defining a new u_str function, the str type is re-defined as unicode for Python 2.6. Then, I only had to update all the strings in the code to use str:

>>> stream.write(str("test"))

Note: I put this code in a file and import it everywhere I need it.

Dictionary keys

The next problem was reported by the 2to3 tool that comes with Py3k.

-                for k in field.keys():
+                for k in list(field.keys()):

Basically, it told me that the dictionary keys() method returns a view in Py3k not a list, so it needs to be converted to a list as explained here:

dict methods dict.keys(), dict.items() and dict.values() return "views" instead of lists. For example, this no longer works: k = d.keys(); k.sort(). Use k = sorted(d) instead (this works in Python 2.5 too and is just as efficient).

Bytes vs. strings

Finally, the last issue was about the difference between strings and bytes in Python 2.x and 3.0. In Python 2.x, bytes is just an alias for str:

>>> bytes
<type 'str'>

In Py3k, bytes and str are different classes and behave differently, see below:

>>> s = "AB"
>>> s[0]
>>> s[1]
>>> b = b"AB"
>>> b[0]
>>> b[1]

This means that one needs to take care of functions returning bytes (e.g. struct.pack) and the operations performed with the returned data, in my case a call to the ord function, that failed with the typical error message:

TypeError: ord() expected string of length 1, but int found

So, following the approaches mentioned above I added the following function to my

def u_ord(c):
    if sys.hexversion >= 0x03000000:
        return c
        return ord(c)

which I used instead of the built-in ord in the struct.pack case.

Hope this helps to someone.

Happy hacking!

14 responses

  1. custom essay writing service says:

    You are point to the topic and well explained each points in a perfect way. I like the way you have written this article. Thank you so much for sharing.. I always love informational post which helps us to increase our knowledge. Here found new post which is best for me.

  2. video making companies says:

    As a matter of first importance, every one of our illustrators, scriptwriters, editors and voice over specialists are professionally prepared staff with years of involvement in their profession. We have effectively overseen a large number of ventures, from recordings for web-based social networking efforts to outlines for sites, so we have a flexible group with an assorted affair to build up your recordings. In addition, we treat every last customer exclusively, ensuring their clasps are interesting with a peculiar touch to emerge from the contenders.

  3. Whiteboard Animation Videos says:

    On account of Python 2.7's novel position as a rendition in the middle of the prior cycles of Python 2 and Python 3.0, it has held on as an exceptionally well known decision for software engineers because of its similarity with numerous hearty libraries. When we discuss Python 2 today, we are ordinarily alluding to the Python 2.7 discharge as that is the most regularly utilized form.

  4. Python Programming Language Assignment Help says:

    The leading assignment help UK firm offers state of the art services to its clients with a promise of delivering all the required work well within the deadline.

  5. Do My Medical Dissertation Writing Service says:

    We also share some information about our business

  6. Joe42 says:

    Something in this post is missing but I do not know what exactly.

  7. Greg45 says:

    Thank you very much for such great blog!

  8. Law essay writer says:

    Python is good for learning the fundamentals for programming without worrying too much about the semantics in a simple interactive environment. It is also good for general purpose scripting web application development and prototyping.

  9. westchester car service says:

    Things are exceptionally open and seriously clear clarification of issues. was genuinely data. Your site is exceptionally valuable.

  10. History Essay Help says:

    Floating point numbers are correct to fifteen decimal locations. Python 2 had separate types for int and long. The int datatype was restricted by sys .Maxint, which numerous by way of platform however turned into typically 232-1. Python 3 has just one integer type, which behaves mostly just like the old long type from python 2.

  11. Write My Assignment for me says:

    In python 2, sorts may be used to symbolize strings. One in all them, str, was a “byte string” type; it represented a chain of bytes in a few unique textual content encoding and defaulted to ascii. The alternative, unicode, became (because the name implies) a unicode string kind. ... Its name is strand it's unicode.

  12. Homework Website says:

    Backward like-minded refers to a hardware or software program system that may use the interface of an older model of the same product. A brand new fashionable product or model is taken into consideration backward well suited while it is able to study, write or view older codecs.

  13. Assignment Writing Service says:

    Things are very open and intensely clear explanation of issues. was truly information. Your website is very beneficial.

  14. BBA Homework Help says:

    Great Information,it has lot for stuff which is informative.I will share the post with my friends.

Leave a Reply