Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

To be fair, proper codepoint processing is a pain even in Java, which was created back when Unicode was in 16-bit mode. Now that it's extended to 32-bits, proper Unicode string looping looks something like this:

    for(int i = 0; i < string.length();) {
        final int codepoint = string.codePointAt(i);
        i += Character.charCount(codepoint);
    }


Actually, that's not correct, and it's the exact same mistake I made when using that API. codePointAt returns the codepoint at index i, where i is measured in 16-bit chars, which means you could index into the middle of a surrogate pair.

The correct version is:

  for (int i = 0; i < string.length(); i = string.offsetByCodePoints(i, 1))
  {
     int codepoint = string.codePointAt(i);
  }
Java 8 seems to have acquired a codePoints() method on the CharSequence interface which seems to do the same thing.

But this just adds to the fact, proper Unicode string processing is a pain :).


I think you missed the part where `i` is not incremented in the for statement, but inside the loop using `Character.charCount`, which returns the number of `char` necessary to represent the code point. If there's something wrong with this, my unit tests have never brought it up, and I am always sure to test with multi-`char` codepoints.


Your right, I did miss it, I apologize. That'll teach me to read code so early in the morning.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: