QTextCodec

Synopsis

Functions

• def canEncode (arg__1)
• def canEncode (arg__1)
• def fromUnicode (in, length[, state=None])
• def fromUnicode (uc)
• def makeDecoder ()
• def makeDecoder (flags)
• def makeEncoder ()
• def makeEncoder (flags)
• def toUnicode (arg__1)
• def toUnicode (chars)
• def toUnicode (in, length[, state=None])

Virtual functions

• def aliases ()
• def convertFromUnicode (in, length, state)
• def convertToUnicode (in, length, state)
• def mibEnum ()
• def name ()

Static functions

• def availableCodecs ()
• def availableMibs ()
• def codecForCStrings ()
• def codecForHtml (ba)
• def codecForHtml (ba, defaultCodec)
• def codecForLocale ()
• def codecForMib (mib)
• def codecForName (name)
• def codecForName (name)
• def codecForTr ()
• def codecForUtfText (ba)
• def codecForUtfText (ba, defaultCodec)
• def setCodecForCStrings (c)
• def setCodecForLocale (c)
• def setCodecForTr (c)
• def validCodecs ()

Detailed Description

The PySide.QtCore.QTextCodec class provides conversions between text encodings.

Qt uses Unicode to store, draw and manipulate strings. In many situations you may wish to deal with data that uses a different encoding. For example, most Japanese documents are still stored in Shift-JIS or ISO 2022-JP, while Russian users often have their documents in KOI8-R or Windows-1251.

Qt provides a set of PySide.QtCore.QTextCodec classes to help with converting non-Unicode formats to and from Unicode. You can also create your own codec classes.

The supported encodings are:

• Apple Roman
• Big5
• Big5-HKSCS
• CP949
• EUC-JP
• EUC-KR
• GB18030-0
• IBM 850
• IBM 866
• IBM 874
• ISO 2022-JP
• ISO 8859-1 to 10
• ISO 8859-13 to 16
• Iscii-Bng, Dev, Gjr, Knd, Mlm, Ori, Pnj, Tlg, and Tml
• JIS X 0201
• JIS X 0208
• KOI8-R
• KOI8-U
• MuleLao-1
• ROMAN8
• Shift-JIS
• TIS-620
• TSCII
• UTF-8
• UTF-16
• UTF-16BE
• UTF-16LE
• UTF-32
• UTF-32BE
• UTF-32LE
• Windows-1250 to 1258
• WINSAMI2

QTextCodecs can be used as follows to convert some locally encoded string to Unicode. Suppose you have some string encoded in Russian KOI8-R encoding, and want to convert it to Unicode. The simple way to do it is like this:

encodedString = QByteArray("...")
codec = QTextCodec.codecForName("KOI8-R")
string = codec.toUnicode(encodedString)

After this, string holds the text converted to Unicode. Converting a string from Unicode to the local encoding is just as easy:

string = u"..."
codec = QTextCodec.codecForName("KOI8-R")
encodedString = codec.fromUnicode(string)

To read or write files in various encodings, use PySide.QtCore.QTextStream and its PySide.QtCore.QTextStream.setCodec() function. See the Codecs example for an application of PySide.QtCore.QTextCodec to file I/O.

Some care must be taken when trying to convert the data in chunks, for example, when receiving it over a network. In such cases it is possible that a multi-byte character will be split over two chunks. At best this might result in the loss of a character and at worst cause the entire conversion to fail.

The approach to use in these situations is to create a PySide.QtCore.QTextDecoder object for the codec and use this PySide.QtCore.QTextDecoder for the whole decoding process, as shown below:

codec = QTextCodec.codecForName("Shift-JIS")
decoder = codec.makeDecoder()

string = u''
while new_data_available():
    chunk = get_new_data()
    string += decoder.toUnicode(chunk)

The PySide.QtCore.QTextDecoder object maintains state between chunks and therefore works correctly even if a multi-byte character is split between chunks.

Creating Your Own Codec Class

Support for new text encodings can be added to Qt by creating PySide.QtCore.QTextCodec subclasses.

The pure virtual functions describe the encoder to the system and the coder is used as required in the different text file formats supported by PySide.QtCore.QTextStream , and under X11, for the locale-specific character input and output.

To add support for another encoding to Qt, make a subclass of PySide.QtCore.QTextCodec and implement the functions listed in the table below.

Function	Description
PySide.QtCore.QTextCodec.name()	Returns the official name for the encoding. If the encoding is listed in the IANA character-sets encoding file , the name should be the preferred MIME name for the encoding.
PySide.QtCore.QTextCodec.aliases()	Returns a list of alternative names for the encoding. PySide.QtCore.QTextCodec provides a default implementation that returns an empty list. For example, “ISO-8859-1” has “latin1”, “CP819”, “IBM819”, and “iso-ir-100” as aliases.
PySide.QtCore.QTextCodec.mibEnum()	Return the MIB enum for the encoding if it is listed in the IANA character-sets encoding file .
PySide.QtCore.QTextCodec.convertToUnicode()	Converts an 8-bit character string to Unicode.
PySide.QtCore.QTextCodec.convertFromUnicode()	Converts a Unicode string to an 8-bit character string.

You may find it more convenient to make your codec class available as a plugin; see How to Create Qt Plugins for details.

See also

PySide.QtCore.QTextStream PySide.QtCore.QTextDecoder PySide.QtCore.QTextEncoder Codecs Example

class PySide.QtCore.QTextCodec

Constructs a PySide.QtCore.QTextCodec , and gives it the highest precedence. The PySide.QtCore.QTextCodec should always be constructed on the heap (i.e. with new ). Qt takes ownership and will delete it when the application terminates.

PySide.QtCore.QTextCodec.ConversionFlag

Constant	Description
QTextCodec.DefaultConversion	No flag is set.
QTextCodec.ConvertInvalidToNull	If this flag is set, each invalid input character is output as a null character.
QTextCodec.IgnoreHeader	Ignore any Unicode byte-order mark and don’t generate any.

PySide.QtCore.QTextCodec.aliases()

Return type:

Subclasses can return a number of aliases for the codec in question.

Standard aliases for codecs can be found in the IANA character-sets encoding file .

static PySide.QtCore.QTextCodec.availableCodecs()

Return type:

Returns the list of all available codecs, by name. Call QTextCodec.codecForName() to obtain the PySide.QtCore.QTextCodec for the name.

The list may contain many mentions of the same codec if the codec has aliases.

See also

PySide.QtCore.QTextCodec.availableMibs() PySide.QtCore.QTextCodec.name() PySide.QtCore.QTextCodec.aliases()

static PySide.QtCore.QTextCodec.availableMibs()

Return type:

Returns the list of MIBs for all available codecs. Call QTextCodec.codecForMib() to obtain the PySide.QtCore.QTextCodec for the MIB.

See also

PySide.QtCore.QTextCodec.availableCodecs() PySide.QtCore.QTextCodec.mibEnum()

PySide.QtCore.QTextCodec.canEncode(arg__1)

Parameters:arg__1 – PySide.QtCore.QChar Return type:PySide.QtCore.bool

Returns true if the Unicode character ch can be fully encoded with this codec; otherwise returns false.

PySide.QtCore.QTextCodec.canEncode(arg__1)

Parameters:arg__1 – unicode Return type:PySide.QtCore.bool

This is an overloaded function.

s contains the string being tested for encode-ability.

static PySide.QtCore.QTextCodec.codecForCStrings()

Return type:PySide.QtCore.QTextCodec

Returns the codec used by PySide.QtCore.QString to convert to and from const char * and QByteArrays. If this function returns 0 (the default), PySide.QtCore.QString assumes Latin-1.

See also

PySide.QtCore.QTextCodec.setCodecForCStrings()

static PySide.QtCore.QTextCodec.codecForHtml(ba, defaultCodec)

Parameters:

• ba – PySide.QtCore.QByteArray
• defaultCodec – PySide.QtCore.QTextCodec

Return type:

PySide.QtCore.QTextCodec

Tries to detect the encoding of the provided snippet of HTML in the given byte array, ba , by checking the BOM (Byte Order Mark) and the content-type meta header and returns a PySide.QtCore.QTextCodec instance that is capable of decoding the html to unicode. If the codec cannot be detected from the content provided, defaultCodec is returned.

See also

PySide.QtCore.QTextCodec.codecForUtfText()

static PySide.QtCore.QTextCodec.codecForHtml(ba)

Parameters:ba – PySide.QtCore.QByteArray Return type:PySide.QtCore.QTextCodec

This is an overloaded function.

Tries to detect the encoding of the provided snippet of HTML in the given byte array, ba , by checking the BOM (Byte Order Mark) and the content-type meta header and returns a PySide.QtCore.QTextCodec instance that is capable of decoding the html to unicode. If the codec cannot be detected, this overload returns a Latin-1 PySide.QtCore.QTextCodec .

static PySide.QtCore.QTextCodec.codecForLocale()

Return type:PySide.QtCore.QTextCodec

Returns a pointer to the codec most suitable for this locale.

On Windows, the codec will be based on a system locale. On Unix systems, starting with Qt 4.2, the codec will be using the iconv library. Note that in both cases the codec’s name will be “System”.

See also

PySide.QtCore.QTextCodec.setCodecForLocale()

static PySide.QtCore.QTextCodec.codecForMib(mib)

Parameters:mib – PySide.QtCore.int Return type:PySide.QtCore.QTextCodec

Returns the PySide.QtCore.QTextCodec which matches the MIBenum mib .

static PySide.QtCore.QTextCodec.codecForName(name)

Parameters:name – str Return type:PySide.QtCore.QTextCodec

Searches all installed PySide.QtCore.QTextCodec objects and returns the one which best matches name ; the match is case-insensitive. Returns 0 if no codec matching the name name could be found.

static PySide.QtCore.QTextCodec.codecForName(name)

Parameters:name – PySide.QtCore.QByteArray Return type:PySide.QtCore.QTextCodec

Searches all installed PySide.QtCore.QTextCodec objects and returns the one which best matches name ; the match is case-insensitive. Returns 0 if no codec matching the name name could be found.

static PySide.QtCore.QTextCodec.codecForTr()

Return type:PySide.QtCore.QTextCodec

Returns the codec used by QObject.tr() on its argument. If this function returns 0 (the default), tr() assumes Latin-1.

See also

PySide.QtCore.QTextCodec.setCodecForTr()

static PySide.QtCore.QTextCodec.codecForUtfText(ba)

Parameters:ba – PySide.QtCore.QByteArray Return type:PySide.QtCore.QTextCodec

This is an overloaded function.

Tries to detect the encoding of the provided snippet ba by using the BOM (Byte Order Mark) and returns a PySide.QtCore.QTextCodec instance that is capable of decoding the text to unicode. If the codec cannot be detected, this overload returns a Latin-1 PySide.QtCore.QTextCodec .

See also

PySide.QtCore.QTextCodec.codecForHtml()

static PySide.QtCore.QTextCodec.codecForUtfText(ba, defaultCodec)

Parameters:

• ba – PySide.QtCore.QByteArray
• defaultCodec – PySide.QtCore.QTextCodec

Return type:

PySide.QtCore.QTextCodec

Tries to detect the encoding of the provided snippet ba by using the BOM (Byte Order Mark) and returns a PySide.QtCore.QTextCodec instance that is capable of decoding the text to unicode. If the codec cannot be detected from the content provided, defaultCodec is returned.

See also

PySide.QtCore.QTextCodec.codecForHtml()

PySide.QtCore.QTextCodec.convertFromUnicode(in, length, state)

Parameters:

• in – PySide.QtCore.QChar
• length – PySide.QtCore.int
• state – PySide.QtCore.QTextCodec::ConverterState

Return type:

PySide.QtCore.QByteArray

PySide.QtCore.QTextCodec subclasses must reimplement this function.

Converts the first number of characters from the input array from Unicode to the encoding of the subclass, and returns the result in a PySide.QtCore.QByteArray .

state can be 0 in which case the conversion is stateless and default conversion rules should be used. If state is not 0, the codec should save the state after the conversion in state , and adjust the remainingChars and invalidChars members of the struct.

PySide.QtCore.QTextCodec.convertToUnicode(in, length, state)

Parameters:

• in – str
• length – PySide.QtCore.int
• state – PySide.QtCore.QTextCodec::ConverterState

Return type:

unicode

PySide.QtCore.QTextCodec subclasses must reimplement this function.

Converts the first len characters of chars from the encoding of the subclass to Unicode, and returns the result in a PySide.QtCore.QString .

state can be 0, in which case the conversion is stateless and default conversion rules should be used. If state is not 0, the codec should save the state after the conversion in state , and adjust the remainingChars and invalidChars members of the struct.

PySide.QtCore.QTextCodec.fromUnicode(in, length[, state=None])

Parameters:

• in – PySide.QtCore.QChar
• length – PySide.QtCore.int
• state – PySide.QtCore.QTextCodec::ConverterState

Return type:

PySide.QtCore.QByteArray

Converts the first number of characters from the input array from Unicode to the encoding of this codec, and returns the result in a PySide.QtCore.QByteArray .

The state of the convertor used is updated.

PySide.QtCore.QTextCodec.fromUnicode(uc)

Parameters:uc – unicode Return type:PySide.QtCore.QByteArray

Converts str from Unicode to the encoding of this codec, and returns the result in a PySide.QtCore.QByteArray .

PySide.QtCore.QTextCodec.makeDecoder(flags)

Parameters:flags – PySide.QtCore.QTextCodec.ConversionFlags Return type:PySide.QtCore.QTextDecoder

PySide.QtCore.QTextCodec.makeDecoder()

Return type:PySide.QtCore.QTextDecoder

Creates a PySide.QtCore.QTextDecoder which stores enough state to decode chunks of char * data to create chunks of Unicode data.

The caller is responsible for deleting the returned object.

PySide.QtCore.QTextCodec.makeEncoder(flags)

Parameters:flags – PySide.QtCore.QTextCodec.ConversionFlags Return type:PySide.QtCore.QTextEncoder

PySide.QtCore.QTextCodec.makeEncoder()

Return type:PySide.QtCore.QTextEncoder

Creates a PySide.QtCore.QTextEncoder which stores enough state to encode chunks of Unicode data as char * data.

The caller is responsible for deleting the returned object.

PySide.QtCore.QTextCodec.mibEnum()

Return type:PySide.QtCore.int

Subclasses of PySide.QtCore.QTextCodec must reimplement this function. It returns the MIBenum (see IANA character-sets encoding file for more information). It is important that each PySide.QtCore.QTextCodec subclass returns the correct unique value for this function.

PySide.QtCore.QTextCodec.name()

Return type:PySide.QtCore.QByteArray

PySide.QtCore.QTextCodec subclasses must reimplement this function. It returns the name of the encoding supported by the subclass.

If the codec is registered as a character set in the IANA character-sets encoding file this method should return the preferred mime name for the codec if defined, otherwise its name.

static PySide.QtCore.QTextCodec.setCodecForCStrings(c)

Parameters:c – PySide.QtCore.QTextCodec

Sets the codec used by PySide.QtCore.QString to convert to and from const char * and QByteArrays. If the codec is 0 (the default), PySide.QtCore.QString assumes Latin-1.

Warning

Some codecs do not preserve the characters in the ASCII range (0x00 to 0x7F). For example, the Japanese Shift-JIS encoding maps the backslash character (0x5A) to the Yen character. To avoid undesirable side-effects, we recommend avoiding such codecs with setCodecsForCString().

See also

PySide.QtCore.QTextCodec.codecForCStrings() PySide.QtCore.QTextCodec.setCodecForTr()

static PySide.QtCore.QTextCodec.setCodecForLocale(c)

Parameters:c – PySide.QtCore.QTextCodec

Set the codec to c ; this will be returned by PySide.QtCore.QTextCodec.codecForLocale() . If c is a null pointer, the codec is reset to the default.

This might be needed for some applications that want to use their own mechanism for setting the locale.

See also

PySide.QtCore.QTextCodec.codecForLocale()

static PySide.QtCore.QTextCodec.setCodecForTr(c)

Parameters:c – PySide.QtCore.QTextCodec

Sets the codec used by QObject.tr() on its argument to c . If c is 0 (the default), tr() assumes Latin-1.

If the literal quoted text in the program is not in the Latin-1 encoding, this function can be used to set the appropriate encoding. For example, software developed by Korean programmers might use eucKR for all the text in the program, in which case the main() function might look like this:

def main():
    app = QApplication([])
    QTextCodec.setCodecForTr(QTextCodec.codecForName("eucKR"))
    ...

Note that this is not the way to select the encoding that the user has chosen. For example, to convert an application containing literal English strings to Korean, all that is needed is for the English strings to be passed through tr() and for translation files to be loaded. For details of internationalization, see Internationalization with Qt .

See also

PySide.QtCore.QTextCodec.codecForTr() PySide.QtCore.QTextCodec.setCodecForCStrings()

PySide.QtCore.QTextCodec.toUnicode(in, length[, state=None])

Parameters:

• in – str
• length – PySide.QtCore.int
• state – PySide.QtCore.QTextCodec::ConverterState

Return type:

unicode

Converts the first size characters from the input from the encoding of this codec to Unicode, and returns the result in a PySide.QtCore.QString .

The state of the convertor used is updated.

PySide.QtCore.QTextCodec.toUnicode(arg__1)

Parameters:arg__1 – PySide.QtCore.QByteArray Return type:unicode

Converts a from the encoding of this codec to Unicode, and returns the result in a PySide.QtCore.QString .

PySide.QtCore.QTextCodec.toUnicode(chars)

Parameters:chars – str Return type:unicode

This is an overloaded function.

chars contains the source characters.

static PySide.QtCore.QTextCodec.validCodecs()

Return type:PySide.QtCore.bool