Unicode Inside

Submitted by lwinmaungmaung on
Unicode Inside

Image removed.

Dr. Nyein Chan Ko Ko
Full Time Technical Expert, Phandeeyar

I am participating both as Phandeeyar and individually. First, when using facebook, there is a new feature such as auto-convert which converts both Zawgyi and Unicodes to readable format both in the Facebook app and Messenger app. They introduced in 2017 August as a beta version. Facebook worked to accelerate unicode progress by communicating with Phandeeyar and take feedback. They also explore for auto-convert technologies. I collect them and send them back. They frequently get feedback and improves their conversion progress.  I also write articles about Myanmar Unicode, ranging from layman to experts. I also prepared for Myanmar Unicode.

Could you explain the pros and cons of Myanmar Unicode?

Starting from pros, Myanmar Unicode is standardized all over the world. There are many technologies supported worldwide. These technologies target Myanmar Unicode. However, we will have to wait for a long time if these technologies have to convert back to our non-standard encoding fonts. It is like the roof of a house. If we haven't prepared correctly, we have to cover with hotfixes. If we changed to a standard form, we can cover and stay convenient. Many problems emerge from both end-users and advanced users such as developers. So, if we change on time, we can use not only released technologies but also new technologies coming soon. In detail, there are very few users who search in the Burmese language. They reported that there are very few contents published with Burmese. We can get more quality content if we are using standardized Myanmar Unicode.  This increased support to Myanmar readers and writers. Using Myanmar Unicode will develop the database storage and machine learning sector.  By standardized data, they are stored and parsed the aim and abstract by analyzing current multiple posts using ML. Our current Zawgyi System can also be used to Machine Learning, but it takes a long run to provide smoothly.

We can state that anybody can resist changing to the new system from the old convenient system. We will face multiple challenges. We can see Myanmar Unicode Transformation progress is working slow and steady. When the government began to push the development, the transition becomes more speedy.

You state that Machine Learning can provide examining our data stored in Myanmar Unicode. There are two statements arise. Who set these standards and how to protect our privacy? What is your opinion on two statements?

Our privacy infringement is nothing concerned directly with Myanmar Unicode transition. Our privacy will have trespassed even if we are using Zawgyi. We are storing our pieces of information on providers servers, including emails and social networks, searching records, etc. These data and information are saved and protected with one statement named privacy policy or agreements. Whatever they wrote, they can get our data quite sure when these agreements do state unclearly. We can see that the advertising methods work by collection not only our texts but also our voices, photos and more. Our data has already breached. I think we don't have to worry about changing progress with leave data security holes.

There is another benefit for using Myanmar Unicode. For example, there is a person who attacks me with a fake account on social media. If I report harassment about me to social media, the management will not have to inspect manually. There will be two steps, machine learning and manual resolving. There are millions of people who are using social media today. The moderator team will not crawl through tons of report tickets one by one. All of the reports have to pass through Machine Learning Progress. These filter records will be scanned manually by the human-powered team. This progress will improve with Myanmar Unicode. 

Another thing is fake news. Social media can detect and remove once they are published using machine learning. Hate speeches can generate toward both sides. Any words that create riots are Hate Speech. If we can predict with Myanmar Unicode and prevent on time, we can save lives.

There are some misconceptions. We are trying to stop fake news. However, some people think using Unicode will prevent authentic news spread. If it is authentic, that news will rise on top. The current situation is we cannot differentiate between fake news and authentic ones. They mixed in our newsfeed. If we have an improved system, we can filter rights from wrongs.

When we are migrating from Zawgyi to Myanmar Unicode, there are tons of oldies. They are created using both Zawgyi and ASCII fonts such as Win Innwa fonts. If a user changed to Myanmar Unicode, how about the accessibility to old systems? Are they production ready?

 

We can categorize in two forms, in personal and business purpose. The solution already exists for both ends.  There are many tools which can convert file names or files written in Zawgyi to Myanmar Unicode. There will be more apps if the user demand still rises. They cannot process when they are working with priority. 

Image removed.

In the Business sector, media websites raise worries about how readers can read their websites after changing font rendering. Now, there is a bunch of tools for embedding fonts on websites, browsers and even database migrations to Unicode. Many tools need a minimum of expertise to migrate to Myanmar Unicode while a few are risky. It will take only some minutes to convert a blog with numerous amount of posts. We can transform from any rendering to Myanmar Unicode. Currently, we target Zawgyi mainly because there are very few websites that use Win fonts. If such sites exist, it can migrate to Myanmar Unicode using tools.

Some people worry about Search Engine Optimization and data loss. They still prefer to use the existing system. How can they make a compatible system to Myanmar Unicode?

So far, there are ways to make current non-standard websites to make visible to Myanmar Unicode visitors. They can install specific fonts to their websites and can update after they migrated to Myanmar Unicode. So, both type of users can read seamlessly. The readers needn't worry about it. 

When a whole website to Myanmar Unicode migrated, there are some normalization process performed. Normalization includes correcting character mistakes. For example, there are some characters such as Zero and Walone are visually same but different. They are corrected when these contents are normalized. When the mistakes are corrected, search engines will show more because the data accuracy increased. There are also cases for invisible keycodes. The normalization process can fix these codes. The readers will not see noticeable changes plus technically correctness will bring more results to search engines. It will bring us improvements.

We are currently using two encoding systems within the same language.  Can we get some information about human hour consumption, CPU hour ruins?

Wastes of hours including Human also CPU cannot calculate easily because this problem arises for a decade. Within this decade, any person except social media users from the countryside suffers from encoding problem. For example, if we send an email, when a user accepts email only with Myanmar Unicode, we have to convert vice versa to communicate. We have to spend at least half to an hour for converting different encodings. Nationwide, a massive amount of Computing and human powers lost to convert different encodings of same language vice versa. The bandwidth cost of downloading fonts wastes our internet bills should also be in the account. To prevent more and more loss before complex forms, we should solve the problem before the time comes by. It is a bottleneck for our community. Many technologies willing to expand but they are waiting for official standards. Myanmar Grammar checker and voice typing, OCR are currently available on Myanmar Unicode. 

May I have any suggestion for Developer about Myanmar Unicode?

Every developer wants customers to use their products. That's why there is a lot of products based on Zawgyi font. Now, Myanmar Unicode Users percentage become larger and larger. The current versions should convert to Myanmar Unicode if it doesn't. Many  Unicode based Repositories are available on GitHub. For example, Myanmar tool is developed by Google and published on Github. Facebook fork and will contribute back. We should also do the same. In my opinion, I feel our community has fewer contributions than others. We share small potion to become a large group. There are many tools required for migration, such as converting phone notes to Myanmar Unicode. Many devices are still working under Android 4.4. Google provides support Myanmar Unicode over android Kitkat. In detail, Unicode 5.1 is in support since Android 2.3. There is a possibility for embedding Myanmar Unicode in older phones by the community.

For End-users, please do not afraid to change to a better system. Please try at first then we will see how easy to transform and use. 

Operating Systems are providing Myanmar Unicode for default however we modified to see non-standard fonts. We don't need to change anything in default. Many vendors change to support Zawgyi to make sales in Myanmar other than default standard Myanmar Unicode.

Many phones are rooted or jailbroken for installing non-standard font. Most people think this is not a severe problem though it is like the key removed from your house main door. Anyone can enter anytime to a rooted or jailbroken phone without any notification. Hackers can steal information and spy on us using rooted phones.  We shouldn't root or jailbreak our phones unless we need to do something more than changing a font. There are very few people root or jailbreak their phones internationally, except geeks. 

How can a community work for migrating Myanmar Unicode?

Some know how to migrate to Myanmar Unicode. Some also volunteer to change to Myanmar Unicode include a large number of phones without any credits. Some do events to help migration. Two things, the first is awareness and support when awareness has spread. It will not succeed if an organization or government only works. Every person within the community should join for the change.

Image removed.Ko Tin Myo Win,
Security Researcher and Bug Bounty Hunter, Wave of Tech

As for free-thinking, I think the movement on changing Zawgyi to Unicode is forceful. My opinion for this movement is it is pushing by force to use Unicode, even if they are interested or not. I agree that Unicode is the standard font. It should be used globally as an international standard. Zawgyi also coexists on its own. There is also another market sector. They process their daily jobs with Old ASCII based fonts instead of Myanmar Unicode and Zawgyi font. They are still using them. Such as Win Innwa fonts and some old ASCII based fonts. Telcos are also pushing via push messages to say that they are using Myanmar Unicode from October 1st. Such a driving force makes end-users confused. All of my family members are now using Myanmar Unicode even they do not notice anything about it. They ask me what is changing to Unicode, what is the progress and I gave the answer, nothing changed. It is already on your phone. Regular users will not know anything about the font or rendering they are using Zawgyi or Unicode except Persons who are technicians. They can move to Zawgyi to Unicode Easily without any notification. When Unicode is pushing as mandatory, the current Zawgyi Users will resist changing.

I use Myanmar Unicode for my daily life. It is convenient but annoyed when I saw forceful promos. Why are they forcing? What will happen when I do not use Myanmar Unicode?

The main downside of Zawgyi is most phones do not support Zawgyi or any Myanmar Unicode in earlier times, including Android 4.0. We have to install it using Rooting or jailbreaking our phones. Later versions of Android and iOS, we can use by default but very few. Rooting an Android or Jailbreaking an iPhone only to change the font is extremely dangerous for user security. I asked many people anything about rooting your phone or jailbreaking. They replied with changing the font purpose only. I don't blame Zawgyi. The shops rooted their phones to install Zawgyi font only, nothing more than that.

Within our point of view, there are two types of separation in Android, system and user storage. When you jailbreak, you are breaking the barrier between the userspace and the system. That means any end-user can access the system partition. That will bring you risks other than profits for many people. We heard much news about phones vulnerables and IoT devices such as smart TVs hacked. The root cause of vulnerable is they are rooted

I want to emphasize that people will use as they wish. The technology side will have to make it compatible. They should not force the user to change anything. Any user who sends using Zawgyi Font will reach to Unicode user with Unicode font without any hassle. Users want to read the text, regardless of Zawgyi or Unicode, they only want to read, except techies. Both solely want they can read and write conveniently. 

Another thing is about end-users who are using Win Innwa or any other ASCII  fonts. Will they be forgotten? There is a large group of people who are using Windows 7 to use Adobe Pagemaker 7. They are using ASCII based fonts. You can see live by visiting Bar Street of Yangon. They are living on Mars while we are fighting each other about changing Myanmar Unicode and Zawgyi. How will you target real users without fighting?

There are problems in other languages. The tech community solves the issues in their ways. In conclusion, this movement should not be rough and perform smoothly over time. We can produce compatible formats around that. Facebook can now auto-convert any text written in both Zawgyi and Myanmar Unicode. If other outsider companies can offer these features and why can't we provide? Other than forcefully pushing, we can reach our target by making compatible things.