RE2 is much faster than the STL regex. According to my tests on simple queries like “.*a.*” at least 15 times. And it is faster than a lot of other regular expressions engines, which potentially can hit exponential runtime. RE2 processes input in a linear time because it uses old good Thompson NFA approach. It does not have new fancy syntax features (like backreferences), but if you can comply with a subset of the traditional Unix egrep regular expression syntax, then you have a tremendous speed win.  Every time you think that you need to integrate regex in your application, I would definitely recommend using RE2. It has a simple and clear interface, wrappers for Python, Ruby, Node.js and more. Supports precompiled regular expressions. Russ Cox did a really great job to make it efficient and robust.

However, it supports only UTF8. There are 2 options on what could be done to “support” UTF-16:

  1. The hardcore way: to rewrite the library. The good starting point will be the patch from the library creator for UCS-2 encoding support.
  2. Straightforward approach: to convert strings. This is obviously easier but will influence performance. Are conversion costs that critical?

Closer look at performance

On my machine (16 GB RAM, Intel Core i7, 2.4 GHz) the speed for allocating and converting 2 mio strings from UTF-16 to UTF-8 on average took 0.62 sec using preallocated buffer. Using a buffer is a good approach if you only need to match an input text, because there is no overhead on creating and deleting strings.

Matching on the same amount of UTF-8 encoded strings for precompiled regex like “.*ab.*txt” took on average 3.09 sec for PartialMatch and 3.49 sec for FullMatch.

The bottom line is: converting UTF-16 encoded strings to UTF-8 adds roughly 20% overhead. For some applications, it could be critical (with real-time performance and large input size), but for the vast majority of others is still OK. That said, depending on the circumstances, you have to decide whether twenty percent will be critical for your application.

Using RE2 with UTF-16 encoded strings
Tagged on:                         

59 thoughts on “Using RE2 with UTF-16 encoded strings

  • February 14, 2019 at 10:39 am
    Permalink

    Thanks for the points you have contributed here. One more thing I would like to convey is that personal computer memory demands generally rise along with other advancements in the technological know-how. For instance, as soon as new generations of processors are brought to the market, there’s usually a similar increase in the dimensions demands of all computer system memory and hard drive space. This is because software program operated by way of these processor chips will inevitably rise in power to make new know-how.

    Reply
  • February 19, 2019 at 6:18 am
    Permalink

    But wanna say that this is very helpful, Thanks for taking your time to write this.

    Reply
  • February 20, 2019 at 10:46 pm
    Permalink

    Very cool. This post is pleasant, my younger sister is totally into this.

    I will convey her for sure. Is it OK to share on Twitter?

    Keep up the great work!

    Reply
  • February 22, 2019 at 8:59 pm
    Permalink

    Following the thesis, you should provide bloodin.essay.amsterdam/coursework/an-essay-about-marriage.html a mini-outline which previews the examples you will use to support loylay.homework.amsterdam/resume-doc/staff-nurse-resume-pdf.html your thesis in the rest of the essay. Not only does this tell the reader what to expect in the paragraphs to come but sala.response.amsterdam/bibliography/voorbeeld-essay-engels.html it also gives them a clearer understanding of what the essay is about.

    Reply
  • February 28, 2019 at 9:33 am
    Permalink

    Thank you a bunch for sharing this with all folks you really understand what you’re speaking approximately! Bookmarked. Please also visit my website =). We will have a link trade arrangement between us!

    Reply
  • February 28, 2019 at 9:33 am
    Permalink

    I have been surfing online more than 3 hours today, yet I never found any interesting article like yours. It’s pretty worth enough for me. In my view, if all site owners and bloggers made good content as you did, the web will be a lot more useful than ever before.

    Reply
  • February 28, 2019 at 9:33 am
    Permalink

    you can also give your baby some antibacterial baby socks to ensure that your baby is always clean.,

    Reply
  • March 4, 2019 at 6:10 pm
    Permalink

    I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post…

    Reply
  • March 13, 2019 at 4:26 am
    Permalink

    Hello! I just now would want to provide a large thumbs up with the wonderful information you’ve got here for this post. We are returning to your blog to get more detailed soon.

    Reply
  • March 16, 2019 at 1:54 am
    Permalink

    Wow, marvelous blog layout! How long have you been blogging for? you make blogging look easy. The overall look of your web site is magnificent, let alone the content!

    Reply
  • March 17, 2019 at 2:48 am
    Permalink

    Awesome! I appreciate your input to this matter. It has been insightful. my blog: how to turn a girl on

    Reply
  • March 28, 2019 at 6:02 am
    Permalink

    Hi there, You have performed a great job. I will certainly digg it and in my opinion recommend to my friends. I am confident they will be benefited from this site.

    Reply
  • April 3, 2019 at 12:41 am
    Permalink

    That is a great tip particularly to those new to the blogosphere. Simple but very precise information… Thanks for sharing this one. A must read article!

    Reply
  • April 8, 2019 at 2:05 pm
    Permalink

    Hey there! I know this is somewhat off topic but I was wondering if you knew where I could locate a captcha plugin for my comment form? I’m using the same blog platform as yours and I’m having trouble finding one? Thanks a lot!

    Reply
  • April 14, 2019 at 7:11 pm
    Permalink

    I and my pals were found to be examining the excellent points located on the blog then all of a sudden I got a horrible feeling I never thanked the blog owner for those strategies. These young men had been totally stimulated to study all of them and have now without a doubt been using these things. Appreciation for indeed being really thoughtful and then for getting certain amazing issues most people are really wanting to learn about. My personal sincere regret for not expressing appreciation to sooner.

    Reply
  • April 16, 2019 at 1:24 pm
    Permalink

    I not to mention my guys appeared to be analyzing the nice points on your website and then unexpectedly came up with a horrible feeling I had not expressed respect to the site owner for those strategies. The ladies were for that reason excited to learn all of them and have in effect undoubtedly been loving them. We appreciate you getting considerably considerate as well as for having this form of decent things millions of individuals are really needing to be aware of. My personal sincere regret for not expressing gratitude to earlier.

    Reply
  • April 18, 2019 at 9:27 am
    Permalink

    I needed to create you that little bit of word to help thank you so much again regarding the magnificent thoughts you have provided on this website. This has been seriously open-handed of you to grant publicly what a few people would have distributed as an e-book to help with making some cash on their own, most notably given that you could possibly have tried it in case you desired. Those solutions additionally worked to be a easy way to be sure that someone else have the same interest just like my very own to understand very much more in terms of this matter. I am sure there are thousands of more pleasurable opportunities in the future for people who scan your website.

    Reply
  • April 20, 2019 at 9:37 am
    Permalink

    I truly wanted to write down a simple remark so as to say thanks to you for some of the precious hints you are showing at this website. My rather long internet look up has at the end been recognized with reasonable ideas to share with my colleagues. I ‘d tell you that most of us site visitors are rather blessed to live in a very good site with very many special individuals with interesting suggestions. I feel rather happy to have seen your webpage and look forward to many more fun times reading here. Thanks a lot once more for a lot of things.

    Reply
  • April 21, 2019 at 4:54 pm
    Permalink

    Nasza apteka zyskala zaufanie juz wielu pacjentow. Dokladamy wszelkich staran, aby zapewnic Wam najlepsza opieke i wsparcie podczas calego procesu realizacji zamowienia. Dostepna u nas Viagra jest oryginalna, a jej zakup jest mozliwy tylko za okazaniem recepty od lekarza. Odpowiadamy na wszystkie pytania naszych klientow, zwiekszajac ich pewnosc, ze dokonali najlepszego wyboru.

    Reply
  • April 28, 2019 at 7:06 am
    Permalink

    I not to mention my friends appeared to be analyzing the nice solutions from the website and all of the sudden developed a terrible suspicion I had not expressed respect to the blog owner for those techniques. All of the women are already for this reason happy to study them and have actually been taking advantage of those things. We appreciate you being indeed kind and also for figuring out certain notable things most people are really needing to understand about. My personal honest regret for not expressing appreciation to earlier.

    Reply
  • April 29, 2019 at 11:06 pm
    Permalink

    I wish to express some thanks to the writer just for rescuing me from this type of dilemma. Because of searching throughout the search engines and seeing suggestions which are not helpful, I thought my life was over. Existing without the solutions to the issues you’ve fixed as a result of your good post is a crucial case, as well as the ones that would have adversely damaged my entire career if I hadn’t come across your blog. Your actual competence and kindness in playing with almost everything was important. I’m not sure what I would’ve done if I hadn’t encountered such a subject like this. I can also at this moment look ahead to my future. Thanks for your time very much for this reliable and amazing help. I won’t think twice to refer your site to anyone who desires guide about this area.

    Reply
  • May 1, 2019 at 8:07 pm
    Permalink

    I am commenting to make you know what a fine discovery my wife’s child developed browsing your blog. She came to understand some pieces, including what it is like to have an awesome helping heart to get many more with no trouble learn about selected complicated topics. You undoubtedly surpassed our expectations. Many thanks for supplying the important, dependable, explanatory and cool tips about this topic to Mary.

    Reply
  • May 2, 2019 at 12:20 pm
    Permalink

    Excellent post! We are linking to this particularly great
    article on our website. Keep up the great writing.

    Reply
  • May 2, 2019 at 10:24 pm
    Permalink

    I want to express thanks to the writer for rescuing me from this particular incident. As a result of looking through the the net and coming across opinions which were not powerful, I assumed my entire life was over. Existing devoid of the approaches to the problems you have sorted out by means of your good write-up is a serious case, and those which might have badly damaged my entire career if I hadn’t discovered your site. Your own personal training and kindness in taking care of all things was very useful. I am not sure what I would’ve done if I hadn’t come across such a thing like this. I can now look forward to my future. Thanks for your time so much for the high quality and amazing guide. I won’t be reluctant to recommend your web site to any person who wants and needs recommendations about this topic.

    Reply
  • May 3, 2019 at 12:22 pm
    Permalink

    My spouse and i ended up being now delighted when Albert managed to carry out his researching because of the precious recommendations he obtained from your very own blog. It is now and again perplexing to just be offering secrets which often many others might have been selling. And we all fully understand we have got the website owner to thank for this. The type of illustrations you have made, the straightforward blog navigation, the relationships you can assist to instill – it’s got everything astounding, and it’s making our son in addition to us know that that concept is interesting, and that is particularly essential. Many thanks for all the pieces!

    Reply
  • May 3, 2019 at 3:02 pm
    Permalink

    It’s fantastic that you are getting thoughts from this paragraph as well as from our discussion made at this time.

    Reply
  • May 4, 2019 at 1:51 am
    Permalink

    Awesome issues here. I’m very glad to peer your article.
    Thanks a lot and I am having a look forward to touch you.
    Will you kindly drop me a e-mail?

    Reply
  • May 5, 2019 at 3:01 am
    Permalink

    Greate pieces. Keep writing such kind of info on your
    site. Im really impressed by it.
    Hello there, You’ve performed a fantastic job.
    I will certainly digg it and for my part recommend to my friends.

    I’m sure they will be benefited from this site.

    Reply
  • May 6, 2019 at 7:20 am
    Permalink

    Hey there I am so excited I found your weblog, I really found you by mistake, while I was browsing on Digg for something else, Anyhow I
    am here now and would just like to say thanks for a
    tremendous post and a all round exciting blog (I also love the theme/design), I don’t have time to read it
    all at the moment but I have bookmarked it and
    also added your RSS feeds, so when I have time I will be back to read a
    great deal more, Please do keep up the great jo.

    Reply
  • May 7, 2019 at 5:55 am
    Permalink

    I wanted to write a comment to express gratitude to you for all of the awesome suggestions you are writing on this website. My extended internet search has at the end been honored with wonderful facts and strategies to go over with my family. I ‘d point out that most of us readers are truly lucky to dwell in a superb community with so many perfect professionals with helpful basics. I feel very fortunate to have discovered your entire webpage and look forward to many more fun minutes reading here. Thanks once again for a lot of things.

    Reply
  • May 7, 2019 at 3:38 pm
    Permalink

    fantastic publish, very informative. I wonder why the opposite specialists of this sector do not notice this.
    You must proceed your writing. I’m sure, you have a great readers’ base already!

    Reply
  • May 7, 2019 at 5:47 pm
    Permalink

    My family members all the time say that I am wasting my time here at net, however I know I am getting know-how all the time by reading such good
    posts.

    Reply
  • May 8, 2019 at 4:18 pm
    Permalink

    Needless expenses, life doesn’t necessarily value your self-esteem, life count on you feel good about by themselves before you can want becoming successful. A lot more no longer realistic, get accustomed to them.

    Reply
  • May 9, 2019 at 3:34 am
    Permalink

    Pretty nice post. I just stumbled upon your blog and wanted
    to say that I’ve truly enjoyed browsing your blog posts.
    In any case I will be subscribing to your rss feed and I hope
    you write again soon!

    Reply
  • May 9, 2019 at 3:22 pm
    Permalink

    I don’t even understand how I ended up here, however I assumed this post was good.
    I don’t realize who you’re however definitely you’re going to
    a famous blogger if you aren’t already. Cheers!

    Reply
  • May 10, 2019 at 4:04 am
    Permalink

    Thanks for sharing your ideas with this blog. Likewise, a misconception regarding the financial institutions intentions while talking about foreclosure is that the financial institution will not getreceive my repayments. There is a certain amount of time in which the bank will require payments here and there. If you are as well deep inside the hole, they may commonly desire that you pay the actual payment in full. However, i am not saying that they will have any sort of repayments at all. When you and the loan company can have the ability to work anything out, your foreclosure process may stop. However, in case you continue to skip payments in the new plan, the foreclosed process can pick up exactly where it left off.

    Reply
  • May 10, 2019 at 4:22 pm
    Permalink

    If some one desires expert view regarding running a blog then i recommend him/her to pay a quick visit this webpage, Keep
    up the fastidious work.

    Reply
  • May 10, 2019 at 7:12 pm
    Permalink

    You actually make it seem so easy with your presentation but I find this topic to be really something that I think I would never understand.
    It seems too complicated and extremely broad for me. I am looking forward for your next post,
    I’ll try to get the hang of it!

    Reply
  • May 11, 2019 at 12:24 am
    Permalink

    Lovely blog! I am loving it!! Will come back again. I am bookmarking your feeds also

    Reply
  • May 11, 2019 at 4:03 am
    Permalink

    I wish to express appreciation to the writer for rescuing me from such a problem. Because of searching through the the web and obtaining thoughts that were not powerful, I thought my life was gone. Existing without the solutions to the difficulties you’ve solved all through your entire review is a critical case, as well as the ones that might have in a negative way affected my career if I hadn’t encountered the website. Your good expertise and kindness in taking care of all the pieces was crucial. I am not sure what I would have done if I hadn’t discovered such a point like this. I’m able to at this point look ahead to my future. Thanks very much for the high quality and amazing help. I will not hesitate to recommend your site to any person who would need guide about this subject matter.

    Reply
  • May 11, 2019 at 11:40 pm
    Permalink

    Everything is very open with a precise explanation of the challenges.
    It was truly informative. Your website is extremely helpful.

    Thanks for sharing!

    Reply
  • May 13, 2019 at 8:51 am
    Permalink

    Hi there, of course this piece of writing is genuinely fastidious and I have learned lot of things from it
    about blogging. thanks.

    Reply
  • May 13, 2019 at 3:36 pm
    Permalink

    magnificent issues altogether, you just won a new reader.
    What could you recommend about your submit that you made a
    few days ago? Any certain?

    Reply
  • May 14, 2019 at 6:55 pm
    Permalink

    Very nice post. I just stumbled upon your blog and wished to say that I have really enjoyed browsing your blog posts.
    After all I will be subscribing to your feed and
    I hope you write again very soon!

    Reply
  • May 15, 2019 at 10:33 am
    Permalink

    I’m really loving the theme/design of your site.
    Do you ever run into any internet browser compatibility problems?
    A number of my blog visitors have complained about
    my website not working correctly in Explorer but looks great in Firefox.
    Do you have any recommendations to help fix this issue?

    Reply
  • May 15, 2019 at 4:57 pm
    Permalink

    Thanks so much for providing individuals with an extraordinarily breathtaking possiblity to read from this website. It is always very pleasing plus packed with amusement for me personally and my office co-workers to search the blog a minimum of thrice in one week to learn the latest items you have got. Not to mention, we are certainly fulfilled with the good strategies served by you. Some 1 areas in this posting are particularly the best we have all had.

    Reply
  • May 15, 2019 at 7:26 pm
    Permalink

    When someone writes an post he/she maintains the idea of a user in his/her brain that how
    a user can be aware of it. So that’s why this piece of writing is great.
    Thanks!

    Reply
  • May 16, 2019 at 2:54 pm
    Permalink

    Howdy fantastic website! Does running a blog like this require a massive amount work?
    I’ve no knowledge of computer programming but I was hoping to start my own blog in the near future.

    Anyway, should you have any ideas or tips for new blog owners please share.
    I understand this is off topic however I simply had to ask.
    Cheers!

    Reply
  • May 16, 2019 at 8:40 pm
    Permalink

    We’re a bunch of volunteers and starting a new scheme in our community.
    Your web site provided us with valuable information to work
    on. You’ve done a formidable job and our whole neighborhood might be grateful to you.

    Reply
  • May 17, 2019 at 6:43 pm
    Permalink

    Good day! Do you know if they make any plugins to help
    with SEO? I’m trying to get my blog to rank for some
    targeted keywords but I’m not seeing very good results.
    If you know of any please share. Thank you!

    Reply
  • May 17, 2019 at 7:33 pm
    Permalink

    When I initially commented I clicked the “Notify me when new comments are added” checkbox and now each time a comment is added I get three emails with the same comment.
    Is there any way you can remove me from that service?
    Thanks a lot!

    Reply

Leave a Reply

Your email address will not be published.