1

As indicated in the documentation, MIMIC-III's patient notes contain so many many newline characters. Why?

Some statistics on MIMIC-III v1.3:

SELECT AVG(CHAR_LENGTH(text)) avg_nb_of_char, 
AVG(CHAR_LENGTH(text) - CHAR_LENGTH(REPLACE(text, '\n', '')) / CHAR_LENGTH('\n')) avg_nb_of_new_lines,
AVG((CHAR_LENGTH(text) / (CHAR_LENGTH(text) - CHAR_LENGTH(REPLACE(text, '\n', '')) / CHAR_LENGTH('\n')))) avg_new_line_every_x_char,

MIN(CHAR_LENGTH(text)) min_nb_of_char, 
MIN(CHAR_LENGTH(text) - CHAR_LENGTH(REPLACE(text, '\n', '')) / CHAR_LENGTH('\n')) min_nb_of_new_lines,
MIN((CHAR_LENGTH(text) / (CHAR_LENGTH(text) - CHAR_LENGTH(REPLACE(text, '\n', '')) / CHAR_LENGTH('\n')))) min_new_line_every_x_char,

MAX(CHAR_LENGTH(text)) max_nb_of_char, 
MAX(CHAR_LENGTH(text) - CHAR_LENGTH(REPLACE(text, '\n', '')) / CHAR_LENGTH('\n')) max_nb_of_new_lines,
MAX((CHAR_LENGTH(text) / (CHAR_LENGTH(text) - CHAR_LENGTH(REPLACE(text, '\n', '')) / CHAR_LENGTH('\n')))) max_new_line_every_x_char,

STD(CHAR_LENGTH(text)) std_nb_of_char, 
STD(CHAR_LENGTH(text) - CHAR_LENGTH(REPLACE(text, '\n', '')) / CHAR_LENGTH('\n')) std_nb_of_new_lines,
STD((CHAR_LENGTH(text) / (CHAR_LENGTH(text) - CHAR_LENGTH(REPLACE(text, '\n', '')) / CHAR_LENGTH('\n')))) std_new_line_every_x_char

FROM mimiciiiv13.noteevents;

returns:

        nb_of_char      nb_of_new_lines         new_line_every_x_char
AVG     1824.8          42.98                   63.2
MIN     3               1                       1.0
MAX     55725           2990                    2753.0
STD     2510.5          67.2                    70.7
  • I don't know that particular format, but I know a few systems that require lines be wrapped at 80 characters. You might want to create a histogram of line lengths, and take a look. (it's possible that the 2753 is actually a error ... or that at some point, they relaxed the standard) – Joe Apr 5 '16 at 16:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.