digitalmars.D.learn - Wired question related with Chinese characters

walker (17/17) Mar 22 2020 I am new to dlang, I like it :)

Steven Schveighoffer (7/29) Mar 22 2020 On windows, the default codepage for the terminal is NOT UTF8. Which

walker (3/10) Mar 22 2020 Thank you for your explanation :)

Adam D. Ruppe (29/30) Mar 22 2020 writeln calls the wrong function for the Windows console.

walker (8/9) Mar 22 2020 On Sunday, 22 March 2020 at 16:12:34 UTC, Adam D. Ruppe wrote:

Adam D. Ruppe (12/14) Mar 22 2020 Terminal.getline *might* work in my lib, but if there's combining

walker (2/5) Mar 23 2020 I have done a small test and It works. Thank you!

lovemini (1/18) Mar 29 2020

lovemini (13/13) Mar 29 2020 import std.stdio;

walker (3/16) Mar 31 2020 Thank you, Chinese output works now, but the Chinese input

walker <growup_wei qq.com> writes:

I am new to dlang, I like it :)
I am on windows10 and use the terminal preview to test and run 
programs.

In order to print Chinese characters correctly, I always use

void main()
{

writeln(var1);
}

I tried dstring but not working. Need Help Here.

---------------------------------
The wired thing is the above approach may not work sometimes -> 
corrupted characters show up.

However, if I run in the same place a little program written in 
nim (just print hello world), then I run the same dlang program, 
the Chinese character is print correctly.

Why that happened?

Mar 22 2020

Steven Schveighoffer <schveiguy gmail.com> writes:

On 3/22/20 11:19 AM, walker wrote:
 I am new to dlang, I like it :)
 I am on windows10 and use the terminal preview to test and run programs.
 
 In order to print Chinese characters correctly, I always use
 
 void main()
 {

 writeln(var1);
 }
 
 I tried dstring but not working. Need Help Here.
 
 ---------------------------------
 The wired thing is the above approach may not work sometimes -> 
 corrupted characters show up.
 
 However, if I run in the same place a little program written in nim 
 (just print hello world), then I run the same dlang program, the Chinese 
 character is print correctly.
 
 Why that happened?

On windows, the default codepage for the terminal is NOT UTF8. Which 
means it may not know how to properly deal with your output.

Most likely nim is making the terminal use the  UTF8 codepage.

See more info here: 
https://superuser.com/questions/269818/change-default-code-page-of-windows-console-to-utf-8

-Steve

Mar 22 2020

walker <growup_wei qq.com> writes:

On Sunday, 22 March 2020 at 15:53:29 UTC, Steven Schveighoffer 
wrote:
 On windows, the default codepage for the terminal is NOT UTF8. 
 Which means it may not know how to properly deal with your 
 output.

 Most likely nim is making the terminal use the  UTF8 codepage.

 See more info here: 
 https://superuser.com/questions/269818/change-default-code-page-of-windows-console-to-utf-8

 -Steve

Thank you for your explanation :)

Mar 22 2020

Adam D. Ruppe <destructionator gmail.com> writes:

On Sunday, 22 March 2020 at 15:19:13 UTC, walker wrote:
 writeln(var1);

writeln calls the wrong function for the Windows console.

You can kinda hack it by changing the code page like Steven said 
(which has other bugs though, but works for many cases), or you 
can call the correct function - WriteConsoleW - yourself instead.

https://docs.microsoft.com/en-us/windows/console/writeconsole

---
import core.sys.windows.windows;

wstring s = ""w; // note it is a wstring, not a string
WriteConsole(GetStdHandle(STD_OUTPUT_HANDLE), s.ptr, cast(DWORD) 
s.length, null, null);
---

my terminal.d library also calls that function but it isn't 
likely to 100% work with Chinese characters either due to other 
bugs. still you can try it if you like

https://github.com/adamdruppe/arsd/blob/master/terminal.d

---
import arsd.terminal;

void main()
{
Terminal terminal = Terminal(ConsoleOutputType.linear);
string var1 = "你好"; // I might have broken this in copy/paste
terminal.writeln(var1); // could work.
}
---

that should work if you have the fonts set up already but just it 
migh have other bugs so I kinda suggest just doing your own write 
function.

readln in the stdlib is similarly broken btw.

Mar 22 2020

walker <growup_wei qq.com> writes:

On Sunday, 22 March 2020 at 16:12:34 UTC, Adam D. Ruppe wrote:

Thank you!

Your arsd.terminal is the first 3rd party module I used, which I 
use to output colored text on the terminal preview on windows10.

I have tried your code above with both string and dstring, It 
works! Really good!

 readln in the stdlib is similarly broken btw.

Which function should I use when I read Chinese characters in the 
terminal?

Mar 22 2020

Adam D. Ruppe <destructionator gmail.com> writes:

On Monday, 23 March 2020 at 01:18:15 UTC, walker wrote:
 Which function should I use when I read Chinese characters in 
 the terminal?

Terminal.getline *might* work in my lib, but if there's combining 
codepoints I'm not sure. You can try it though and let me know if 
you are already using the lib.

There's also ReadConsole in core.sys.windows.windows that isn't a 
bad experience (just Windows only). It works with `wstring` just 
like WriteConsole and is the native function with unicode 
support. The operating system also manages some editing for the 
user, cursor placement, etc. so is more likely to work better 
with edge cases than my library's custom implementation of all 
that.

https://docs.microsoft.com/en-us/windows/console/readconsole

Mar 22 2020

walker <growup_wei qq.com> writes:

On Monday, 23 March 2020 at 01:40:16 UTC, Adam D. Ruppe wrote:

 Terminal.getline *might* work in my lib, but if there's 
 combining codepoints I'm not sure. You can try it though and 
 let me know if you are already using the lib.

I have done a small test and It works. Thank you!

Mar 23 2020

lovemini <lovemini gmail.com> writes:

On Sunday, 22 March 2020 at 15:19:13 UTC, walker wrote:
 I am new to dlang, I like it :)
 I am on windows10 and use the terminal preview to test and run 
 programs.

 In order to print Chinese characters correctly, I always use

 void main()
 {

 writeln(var1);
 }

 I tried dstring but not working. Need Help Here.

 ---------------------------------
 The wired thing is the above approach may not work sometimes -> 
 corrupted characters show up.

 However, if I run in the same place a little program written in 
 nim (just print hello world), then I run the same dlang 
 program, the Chinese character is print correctly.

 Why that happened?

Mar 29 2020

lovemini <lovemini gmail.com> writes:

import std.stdio;

void main()
{
	version( Windows ) {
		//直接运行中文显示乱码,原因在于Windows控制台默认编码为
936，而D语言输出utf-8
		//可以将控制台编码修改为 utf-8,命令为 "CHCP 65001"
		//修改后就可以显示中文了
		import core.sys.windows.windows;
		SetConsoleCP(65001);
		SetConsoleOutputCP( 65001 );
	}

     writeln("Hello World! 你好，中国！");
}

Mar 29 2020

walker <growup_wei qq.com> writes:

On Sunday, 29 March 2020 at 10:36:53 UTC, lovemini wrote:
 import std.stdio;

 void main()
 {
 	version( Windows ) {
 		//直接运行中文显示乱码,原因在于Windows控制台默认编码为
936，而D语言输出utf-8
 		//可以将控制台编码修改为 utf-8,命令为 "CHCP 65001"
 		//修改后就可以显示中文了
 		import core.sys.windows.windows;
 		SetConsoleCP(65001);
 		SetConsoleOutputCP( 65001 );
 	}

     writeln("Hello World! 你好，中国！");
 }

Thank you, Chinese output works now, but the Chinese input 
doesn't work. How to change input behaviors?

Mar 31 2020

D Programming

C/C++ Programming

Other

digitalmars.D.learn - Wired question related with Chinese characters