digitalmars.D - MBCS character code support

Katayama Hirofumi MZ (6/6) May 15 2012 Hello, everyone!

Katayama Hirofumi MZ (65/65) May 15 2012 You can convert UTF-8 to Shift_JIS by the following code.
Katayama Hirofumi MZ (3/3) May 15 2012 All Japaneses and/or other Asians want native MBCS support.

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= (6/8) May 15 2012 I really do not understand why you want to use Shift-JIS. Unicode has

Katayama Hirofumi MZ (7/10) May 15 2012 On Windows 9x, there is no Unicode support. Instead, native MBCS

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= (6/14) May 15 2012 D does not support Windows versions older than Windows 2000.

Denis Shelomovskij (7/19) May 19 2012 D2 has no Windows 2000 support for a long time.

Kagamin (3/4) May 16 2012 http://msdn.microsoft.com/en-us/goglobal/bb688166

Walter Bright (6/14) May 16 2012 I'm familiar with Shift-JIS from the C compiler days.

Katayama Hirofumi MZ (1/1) May 20 2012 Can D convert strings on compile time?

Walter Bright (2/3) May 20 2012 Yes, you can write a CTFE function to do it.
Katayama Hirofumi MZ (3/3) May 23 2012 I made source code converter for Shift_JIS.

"Katayama Hirofumi MZ" <katayama.hirofumi.mz gmail.com> writes:

Hello, everyone!

I want multibyte character string (MBCS) support except Unicode, UTF-8 and 
UTF-16.

Could you make the D compiler generate Shift_JIS, EUC-JP code for every 
string literals by a specific command line option?

Shift_JIS and EUC-JP are Japanese character set.

May 15 2012

"Katayama Hirofumi MZ" <katayama.hirofumi.mz gmail.com> writes:

You can convert UTF-8 to Shift_JIS by the following code.


/* Linux, FreeBSD or UNIX */
#include <iconv.h>
iconv_t g_icUTF8toSJIS;

char *convert_utf8_to_sjis(char *in)
{
     char *out, *p_in, *p_out,
     size_t in_size, out_size;

     in_size = strlen(in);
     out_size = in_size;
     out = (char *)malloc(out_size + 1);
     if (out == NULL)
         return NULL;

     p_in = in;
     p_out = out;
     iconv(g_icUTF8toSJIS, &p_in, &in_size, &p_out, &out_size);
     *p_out = 0;

     return out;
}


int main(void)
{
     char *out;
     g_icUTF8toSJIS = iconv_open("UTF-8", "SJIS");
     if (g_icUTF8toSJIS == (iconv_t)-1) {
         // error
     }
     ...
     out = convert_utf8_to_sjis(...);
     ...
     free(out);
     ...

     iconv_close(g_icUTF8toSJIS);
     return 0;
}

/* Windows */
#include <windows.h>

char *UTF8toSJIS(char *utf8)
{
     char *wide, *sjis;
     int size;

     size = MultiByteToWideChar(CP_UTF8, 0, utf8, -1, 0, 0);
     wide = (char *)malloc((size + 1) * sizeof(WCHAR));
     if (wide == NULL)
         return NULL;
     MultiByteToWideChar(CP_UTF8, 0, utf8, -1, wide, size);

     size = WideCharToMultiByte(CP_ACP, 0, wide, -1, 0, 0, 0, 0);
     sjis = malloc(size * 2 + 1);
     if (sjis == NULL) {
         free(wide);
         return NULL;
     }

     WideCharToMultiByte(CP_ACP, 0, wide, -1, sjis, size, 0, 0);
     free(wide);

     return sjis;
}

int main(void)
{
     char *out;
     ...
     out = UTF8toSJIS(...);
     ...
     free(out);
     ...
     return 0;
}

May 15 2012

"Katayama Hirofumi MZ" <katayama.hirofumi.mz gmail.com> writes:

All Japaneses and/or other Asians want native MBCS support.
Please let the D compiler generate Shift_JIS code for literal 
strings.

May 15 2012

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <xtzgzorex gmail.com> writes:

On 16-05-2012 06:04, Katayama Hirofumi MZ wrote:
 All Japaneses and/or other Asians want native MBCS support.
 Please let the D compiler generate Shift_JIS code for literal strings.

I really do not understand why you want to use Shift-JIS. Unicode has 
long superseded all these magical encodings used all over the world. Why 
oppose a unified encoding?

-- 
- Alex

May 15 2012

"Katayama Hirofumi MZ" <katayama.hirofumi.mz gmail.com> writes:

On Wednesday, 16 May 2012 at 04:12:04 UTC, Alex Rønne Petersen 
wrote:
 I really do not understand why you want to use Shift-JIS. 
 Unicode has long superseded all these magical encodings used 
 all over the world. Why oppose a unified encoding?

On Windows 9x, there is no Unicode support. Instead, native MBCS
encoding exists.

So, if the D Windows program could use UTF-8 only, then the 
programmer
should let the D program convert these strings to Shift_JIS.

May 15 2012

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 16-05-2012 06:18, Katayama Hirofumi MZ wrote:
 On Wednesday, 16 May 2012 at 04:12:04 UTC, Alex Rønne Petersen wrote:
 I really do not understand why you want to use Shift-JIS. Unicode has
 long superseded all these magical encodings used all over the world.
 Why oppose a unified encoding?

 On Windows 9x, there is no Unicode support. Instead, native MBCS
 encoding exists.

 So, if the D Windows program could use UTF-8 only, then the programmer
 should let the D program convert these strings to Shift_JIS.

D does not support Windows versions older than Windows 2000.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

May 15 2012

Denis Shelomovskij <verylonglogin.reg gmail.com> writes:

16.05.2012 8:26, Alex Rønne Petersen написал:
 On 16-05-2012 06:18, Katayama Hirofumi MZ wrote:
 On Wednesday, 16 May 2012 at 04:12:04 UTC, Alex Rønne Petersen wrote:
 I really do not understand why you want to use Shift-JIS. Unicode has
 long superseded all these magical encodings used all over the world.
 Why oppose a unified encoding?

 On Windows 9x, there is no Unicode support. Instead, native MBCS
 encoding exists.

 So, if the D Windows program could use UTF-8 only, then the programmer
 should let the D program convert these strings to Shift_JIS.

 D does not support Windows versions older than Windows 2000.

D2 has no Windows 2000 support for a long time.

http://d.puremagic.com/issues/show_bug.cgi?id=6024
https://github.com/D-Programming-Language/druntime/pull/212

-- 
Денис В. Шеломовский
Denis V. Shelomovskij

May 19 2012

"Kagamin" <spam here.lot> writes:

On Wednesday, 16 May 2012 at 04:19:00 UTC, Katayama Hirofumi MZ 
wrote:
 On Windows 9x, there is no Unicode support.

http://msdn.microsoft.com/en-us/goglobal/bb688166

May 16 2012

Walter Bright <newshound2 digitalmars.com> writes:

On 5/15/2012 7:12 PM, Katayama Hirofumi MZ wrote:
 Hello, everyone!
 
 I want multibyte character string (MBCS) support except Unicode, UTF-8 and
UTF-16.
 
 Could you make the D compiler generate Shift_JIS, EUC-JP code for every string 
 literals by a specific command line option?
 
 Shift_JIS and EUC-JP are Japanese character set.

I'm familiar with Shift-JIS from the C compiler days.

D is designed to internally be all UTF-8, and the runtime code all assumes
UTF-8. I recommend programming in such a way that user input is converted from
Shift-JIS to UTF-8, then all processing is done in terms of UTF-8, then the
output is converted to Shift-JIS.

May 16 2012

"Katayama Hirofumi MZ" <katayama.hirofumi.mz gmail.com> writes:

Can D convert strings on compile time?

May 20 2012

Walter Bright <newshound2 digitalmars.com> writes:

On 5/20/2012 10:14 PM, Katayama Hirofumi MZ wrote:
 Can D convert strings on compile time?

Yes, you can write a CTFE function to do it.

May 20 2012

"Katayama Hirofumi MZ" <katayama.hirofumi.mz gmail.com> writes:

I made source code converter for Shift_JIS.
Settled.

http://ime.nu/katahiromz.web.fc2.com/d/mbconvd.html

May 23 2012

D Programming

C/C++ Programming

Other

digitalmars.D - MBCS character code support