Skip to content

Server-side unicode support (normalization, removing accents) #3174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
DartBot opened this issue May 22, 2012 · 7 comments
Open

Server-side unicode support (normalization, removing accents) #3174

DartBot opened this issue May 22, 2012 · 7 comments
Labels
area-core-library SDK core library issues (core, async, ...); use area-vm or area-web for platform specific libraries. core-n P2 A bug or feature request we're likely to work on type-enhancement A request for a change that isn't a bug

Comments

@DartBot
Copy link

DartBot commented May 22, 2012

This issue was originally filed by [email protected]

In Java, I can easily remove accents from strings like this:
 Normalizer.normalize(text, Form.NFD).replaceAll("\p{InCombiningDiacriticalMarks}+", "")

My primary use case is to generate URL form user-supplied content (e.g. title).

It would be nice if we could use similar feature in Dart on the server-side. Client-side unicode support in the javascript world might be too large.
@peter-ahe-google
Copy link
Contributor

Added Area-Library, Triaged labels.

@lrhn
Copy link
Member

lrhn commented Aug 19, 2013

Removed Type-Defect label.
Added Type-Enhancement label.

@sgjesse
Copy link
Contributor

sgjesse commented Jan 2, 2014

Issue #7611 has been merged into this issue.

@DartBot DartBot added Type-Enhancement area-core-library SDK core library issues (core, async, ...); use area-vm or area-web for platform specific libraries. labels Jan 2, 2014
@DartBot DartBot mentioned this issue Jun 4, 2015
Closed
@kevmoo kevmoo added P2 A bug or feature request we're likely to work on type-enhancement A request for a change that isn't a bug and removed triaged labels Feb 29, 2016
@lrhn lrhn added the core-m label Aug 11, 2017
@floitschG floitschG added core-n and removed core-m labels Aug 31, 2017
@natebosch
Copy link
Member

Is this covered by dart-lang/language#49 ?

@yshrsmz
Copy link

yshrsmz commented Dec 10, 2018

JFYI, I have released unorm_dart which might resolve some of the issue described here. Hope it helps.

@zoechi zoechi mentioned this issue Feb 7, 2019
Closed
@cassioseffrin
Copy link

There is an amazing package called diacritic: https://pub.dev/packages/diacritic

import 'package:diacritic/diacritic.dart';

void main() {
print(removeDiacritics('árvíztűrő tükörfúrógép'));
}

outputs: 'arvizturo tukorfurogep'

copybara-service bot pushed a commit that referenced this issue Oct 15, 2021
@sigurdm
Bump pub to 35681b0126a1fb48bf2062dd09f74296715402c2
c176084 
The pub client still won't send analytics with this PR. (This will be
enabled in a follow-up).

Changes:
```
> git log --format="%C(auto) %h %s" 0764437..35681b0
 https://dart.googlesource.com/pub.git/+/35681b01 Analytics (#2778)
 https://dart.googlesource.com/pub.git/+/9c65d31c Accept 'platforms' property in pubspec.yaml (#3176)
 https://dart.googlesource.com/pub.git/+/1af0de54 Accept 'screenshots' property in pubspec.yaml (#3174)
 https://dart.googlesource.com/pub.git/+/19045b95 Tests related to token authentication (#3147)
 https://dart.googlesource.com/pub.git/+/f36da07f Use applicationConfigHome from package:cli_util (#3164)

```

Diff: https://dart.googlesource.com/pub.git/+/0764437088fd58eb7af779ecef66bab40dfcf2e9~..35681b0126a1fb48bf2062dd09f74296715402c2/
Change-Id: I7313f3125934aba3473b2725074bfc7fd92e25e4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217004
Reviewed-by: Jonas Jensen <[email protected]>
Commit-Queue: Sigurd Meldgaard <[email protected]>
@iulian0512
Copy link

+1 the ICU libraries are the gold standard for normalization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-core-library SDK core library issues (core, async, ...); use area-vm or area-web for platform specific libraries. core-n P2 A bug or feature request we're likely to work on type-enhancement A request for a change that isn't a bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@kevmoo @yshrsmz @sgjesse @lrhn @peter-ahe-google @natebosch @floitschG @DartBot @cassioseffrin @iulian0512