On Wednesday, July 20, 2011 03:23:58 PM Sean Carolan wrote: [snip]

Where did all the letter n's go?

I can't duplicate the problem here on a CentOS 5.6 box. What locale are you set to? Here's what I get (note that a copy from the e-mail you sent embedded newlines, which had to be stripped out (one of the many things xargs makes trivially easy) to get the result....):

$ cat loremipsum.txt Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec quis ipsum sed elit laoreet malesuada. Quisque rhoncus dui vitae eros euismod fermentum sollicitudin sem scelerisque. Nulla facilisi. Maecenas mollis pulvinar euismod. Duis viverra pharetra turpis eget feugiat. Nulla facilisi. Nullam facilisis, felis vitae lacinia fermentum, enim erat placerat erat, vel imperdiet lorem velit et ligula. Nam congue est in nisl lacinia lobortis. Vivamus elementum lacinia sodales. Curabitur commodo risus tincidunt augue pulvinar vehicula. Morbi eget velit sollicitudin nibh porta molestie. Maecenas in augue id quam ullamcorper rutrum.

$ myvar=$(grep lorem loremipsum.txt) $ echo $myvar fermentum, enim erat placerat erat, vel imperdiet lorem velit et $ mv loremipsum.txt lorem-newlines.txt $ cat lorem-newlines.txt |xargs -n 100 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec quis ipsum sed elit laoreet malesuada. Quisque rhoncus dui vitae eros euismod fermentum sollicitudin sem scelerisque. Nulla facilisi. Maecenas mollis pulvinar euismod. Duis viverra pharetra turpis eget feugiat. Nulla facilisi. Nullam facilisis, felis vitae lacinia fermentum, enim erat placerat erat, vel imperdiet lorem velit et ligula. Nam congue est in nisl lacinia lobortis. Vivamus elementum lacinia sodales. Curabitur commodo risus tincidunt augue pulvinar vehicula. Morbi eget velit sollicitudin nibh porta molestie. Maecenas in augue id quam ullamcorper rutrum. $ cat lorem-newlines.txt |xargs -n 100 >loremipsum.txt $ myvar=$(grep lorem loremipsum.txt) $ echo $myvar Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec quis ipsum sed elit laoreet malesuada. Quisque rhoncus dui vitae eros euismod fermentum sollicitudin sem scelerisque. Nulla facilisi. Maecenas mollis pulvinar euismod. Duis viverra pharetra turpis eget feugiat. Nulla facilisi. Nullam facilisis, felis vitae lacinia fermentum, enim erat placerat erat, vel imperdiet lorem velit et ligula. Nam congue est in nisl lacinia lobortis. Vivamus elementum lacinia sodales. Curabitur commodo risus tincidunt augue pulvinar vehicula. Morbi eget velit sollicitudin nibh porta molestie. Maecenas in augue id quam ullamcorper rutrum. $

Sean Carolan wrote:

This is kind of odd.

[scarolan@host:~]$ cat loremipsum.txt Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec quis ipsum sed elit laoreet malesuada. Quisque rhoncus dui vitae eros euismod fermentum sollicitudin sem scelerisque. Nulla facilisi. Maecenas mollis pulvinar euismod. Duis viverra pharetra turpis eget feugiat. Nulla facilisi. Nullam facilisis, felis vitae lacinia fermentum, enim erat placerat erat, vel imperdiet lorem velit et ligula. Nam congue est in nisl lacinia lobortis. Vivamus elementum lacinia sodales. Curabitur commodo risus tincidunt augue pulvinar vehicula. Morbi eget velit sollicitudin nibh porta molestie. Maecenas in augue id quam ullamcorper rutrum.

[scarolan@host:~]$ vi loremipsum.txt [scarolan@host:~]$ myvar=$(grep lorem loremipsum.txt) [scarolan@host:~]$ echo $myvar Lorem ipsum dolor sit amet, co sectetur adipisci g elit. Do ec quis ipsum sed elit laoreet malesuada. Quisque rho cus dui vitae eros euismod ferme tum sollicitudi sem scelerisque. Nulla facilisi. Maece as mollis pulvi ar euismod. Duis viverra pharetra turpis eget feugiat. Nulla facilisi. Nullam facilisis, felis vitae laci ia ferme tum, e im erat placerat erat, vel imperdiet lorem velit et ligula. Nam co gue est i isl laci ia lobortis. Vivamus eleme tum laci ia sodales. Curabitur commodo risus ti cidu t augue pulvi ar vehicula. Morbi eget velit sollicitudi ibh porta molestie. Maece as i augue id quam ullamcorper rutrum.

Where did all the letter n's go?

1. What does "grep lorem loremipsum.txt" say? 2. What does "echo loremipsum.txt | grep lorem" say?

--On Wednesday, July 20, 2011 02:53:06 PM -0500 Sean Carolan scarolan@gmail.com wrote:

Now the next question would be, how can I include the \n characters in my variable string, without fudging with $IFS?

Can you describe, functionally and a bit higher level, what you're trying to achieve? Is myvar supposed to contain the entire file content if the file contains "lorem", or just any lines containing lorem complete with newline, or are you expecting some other form of delimiter, or what? Also of interest is what you're going to do with myvar after the fact (write it out, do a replacement, whatever).

For example, if you're trying to do some form of processing that only considers lines containing lorem, other tools might give a cleaner, faster, and more maintainable result:

#! /usr/bin/perl use taint; use strict;

while (<>) { if (m,lorem,) { # do something with $_ } }

(No, I don't advocate perl for everything, but knowing more about the problem can help in determining a suitable solution.)

Devin

On Wednesday 20 Jul 2011 20:53:06 Sean Carolan wrote:

I had this in my bash history:

IFS='\n'

That seems to have been the cause of the missing n's.

I know you used python in the end, but for the record this should have been:

IFS=$'\n'

i.e. the \n has to be expanded to a newline, otherwise it causes word splitting on either '' or 'n'.