I am writing a long bash script under CentOS 7 where perl is used for manipulating some external files. So far I am using perl one-liners to do so but ran into a problem when I need to append text to an external file.
Here is a simplified example in the bash script where txt is a bash variable which I built containing a longish text with multiple newlines:
txt="a b$'\n'cd ef$'\n'g h$'\n'ij kl"
A simplified perl one-liner to append the text in the variable above to some file in the bash script would be:
perl -pe 'eof && do{print $_'"${txt}"'; exit}' someexternalfile.txt
This works when fine when $txt does /not/ contain any spaces but falls apart when it does.
I would like to keep the above structure, ie using bash variables to build text strings and one-liners to do the text manipulation. Hopefully there is a "simple" solution to do this, I have tried many variations and failed miserably... Note that I also want to use a similar pattern to do substitutions in external files, I would thus like to use the same code pattern.
Thanks.
On 2022-01-30 20:12, H wrote:
I am writing a long bash script under CentOS 7 where perl is used for manipulating some external files. So far I am using perl one-liners to do so but ran into a problem when I need to append text to an external file.
Rewrite the whole thing in Perl. Over the years, I have rewritten scores of bash scripts in Perl, and they 1. execute much faster, 2. don't lose variables from while loops, 3. can use any of the thousands of CPAN modules, 4. can use libraries, and on and on.
Todd Merriman Software Toolz
On January 30, 2022 8:25:46 PM EST, mailist mailist@toolz.com wrote:
On 2022-01-30 20:12, H wrote:
I am writing a long bash script under CentOS 7 where perl is used for manipulating some external files. So far I am using perl one-liners
to
do so but ran into a problem when I need to append text to an
external
file.
Rewrite the whole thing in Perl. Over the years, I have rewritten scores of bash scripts in Perl, and they 1. execute much faster, 2. don't lose variables from while loops, 3. can use any of the thousands of CPAN modules, 4. can use libraries, and on and on.
Todd Merriman Software Toolz _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Sorry but not in the plans. Not needed.
On 1/30/22 18:12, H wrote:
I am writing a long bash script under CentOS 7 where perl is used for manipulating some external files. So far I am using perl one-liners to do so but ran into a problem when I need to append text to an external file.
Here is a simplified example in the bash script where txt is a bash variable which I built containing a longish text with multiple newlines:
txt="a b$'\n'cd ef$'\n'g h$'\n'ij kl"
A simplified perl one-liner to append the text in the variable above to some file in the bash script would be:
perl -pe 'eof && do{print $_'"${txt}"'; exit}' someexternalfile.txt
This works when fine when $txt does /not/ contain any spaces but falls apart when it does.
I would like to keep the above structure, ie using bash variables to build text strings and one-liners to do the text manipulation. Hopefully there is a "simple" solution to do this, I have tried many variations and failed miserably... Note that I also want to use a similar pattern to do substitutions in external files, I would thus like to use the same code pattern.
I don't understand why:
echo -e $txt >> someexternalfile.txt
doesn't do what you want, or if perl is absolutely what you need:
perl -e "print "${txt}";" >> someexternalfile.txt
I have no idea if you are trying to output literal $'s or 's or not.
On 01/30/2022 11:00 PM, Orion Poplawski wrote:
On 1/30/22 18:12, H wrote:
I am writing a long bash script under CentOS 7 where perl is used for manipulating some external files. So far I am using perl one-liners to do so but ran into a problem when I need to append text to an external file.
Here is a simplified example in the bash script where txt is a bash variable which I built containing a longish text with multiple newlines:
txt="a b$'\n'cd ef$'\n'g h$'\n'ij kl"
A simplified perl one-liner to append the text in the variable above to some file in the bash script would be:
perl -pe 'eof && do{print $_'"${txt}"'; exit}' someexternalfile.txt
This works when fine when $txt does /not/ contain any spaces but falls apart when it does.
I would like to keep the above structure, ie using bash variables to build text strings and one-liners to do the text manipulation. Hopefully there is a "simple" solution to do this, I have tried many variations and failed miserably... Note that I also want to use a similar pattern to do substitutions in external files, I would thus like to use the same code pattern.
I don't understand why:
echo -e $txt >> someexternalfile.txt
doesn't do what you want, or if perl is absolutely what you need:
perl -e "print "${txt}";" >> someexternalfile.txt
I have no idea if you are trying to output literal $'s or 's or not.
Thank you, it works! I had forgotten to escape the quotes around my bash variable...
On 01/31/2022 09:59 PM, H wrote:
On 01/30/2022 11:00 PM, Orion Poplawski wrote:
On 1/30/22 18:12, H wrote:
I am writing a long bash script under CentOS 7 where perl is used for manipulating some external files. So far I am using perl one-liners to do so but ran into a problem when I need to append text to an external file.
Here is a simplified example in the bash script where txt is a bash variable which I built containing a longish text with multiple newlines:
txt="a b$'\n'cd ef$'\n'g h$'\n'ij kl"
A simplified perl one-liner to append the text in the variable above to some file in the bash script would be:
perl -pe 'eof && do{print $_'"${txt}"'; exit}' someexternalfile.txt
This works when fine when $txt does /not/ contain any spaces but falls apart when it does.
I would like to keep the above structure, ie using bash variables to build text strings and one-liners to do the text manipulation. Hopefully there is a "simple" solution to do this, I have tried many variations and failed miserably... Note that I also want to use a similar pattern to do substitutions in external files, I would thus like to use the same code pattern.
I don't understand why:
echo -e $txt >> someexternalfile.txt
doesn't do what you want, or if perl is absolutely what you need:
perl -e "print "${txt}";" >> someexternalfile.txt
I have no idea if you are trying to output literal $'s or 's or not.
Thank you, it works! I had forgotten to escape the quotes around my bash variable...
I am still having a problem. The following (where $txt is an arbitrary string) works:
perl -e 'print '""${txt}""';'
The following does not work (I want to append the content of the $txt to the end of an existing file in-place):
perl -i -pe 'eof && do{print $_''"aaa"''; exit}' somefile.txt
but this does:
perl -i -pe "eof && do{print $_"""${txt}""'; exit}' somefile.txt
as does:
perl -i -pe "eof && do{print $_"""${txt}"""; exit}" somefile.txt
The difference is that the last two perl command strings use " rather than '.
My questions are:
- Why would not using single-quotes for parts of the perl command string work?
- Is there any reason I should fight this or should I just go with double-quotes for all parts of the perl command string? Any downside? Remember, these are all in bash scripts and I am looking for a "pattern" to use for other, more complicated text substitutions, hence the use of perl.
Thank you!
On 02/02/2022 08:54 PM, H wrote:
On 01/31/2022 09:59 PM, H wrote:
On 01/30/2022 11:00 PM, Orion Poplawski wrote:
On 1/30/22 18:12, H wrote:
I am writing a long bash script under CentOS 7 where perl is used for manipulating some external files. So far I am using perl one-liners to do so but ran into a problem when I need to append text to an external file.
Here is a simplified example in the bash script where txt is a bash variable which I built containing a longish text with multiple newlines:
txt="a b$'\n'cd ef$'\n'g h$'\n'ij kl"
A simplified perl one-liner to append the text in the variable above to some file in the bash script would be:
perl -pe 'eof && do{print $_'"${txt}"'; exit}' someexternalfile.txt
This works when fine when $txt does /not/ contain any spaces but falls apart when it does.
I would like to keep the above structure, ie using bash variables to build text strings and one-liners to do the text manipulation. Hopefully there is a "simple" solution to do this, I have tried many variations and failed miserably... Note that I also want to use a similar pattern to do substitutions in external files, I would thus like to use the same code pattern.
I don't understand why:
echo -e $txt >> someexternalfile.txt
doesn't do what you want, or if perl is absolutely what you need:
perl -e "print "${txt}";" >> someexternalfile.txt
I have no idea if you are trying to output literal $'s or 's or not.
Thank you, it works! I had forgotten to escape the quotes around my bash variable...
I am still having a problem. The following (where $txt is an arbitrary string) works:
perl -e 'print '""${txt}""';'
The following does not work (I want to append the content of the $txt to the end of an existing file in-place):
perl -i -pe 'eof && do{print $_''"aaa"''; exit}' somefile.txt
but this does:
perl -i -pe "eof && do{print $_"""${txt}""'; exit}' somefile.txt
as does:
perl -i -pe "eof && do{print $_"""${txt}"""; exit}" somefile.txt
The difference is that the last two perl command strings use " rather than '.
My questions are:
Why would not using single-quotes for parts of the perl command string work?
Is there any reason I should fight this or should I just go with double-quotes for all parts of the perl command string? Any downside? Remember, these are all in bash scripts and I am looking for a "pattern" to use for other, more complicated text substitutions, hence the use of perl.
Thank you!
I see I made a mistake, the line:
perl -i -pe 'eof && do{print $_''"aaa"''; exit}' somefile.txt
should be:
perl -i -pe 'eof && do{print $_''"${txt}"''; exit}' somefile.txt
Related question, if the $txt string contains eg $ or another special character, what would be the best way of escaping it so it is not substituted by perl?
Thank you.
I avoid using ' or " in one-liners or even programs.
I use q() or qq().
These are quote-like operators for single quote and double quote.
There are, of course, many more:
https://perldoc.perl.org/perlop#Quote-and-Quote-like-Operators
gizmo
On 02/03/2022 08:39 AM, Joe Kline wrote:
I avoid using ' or " in one-liners or even programs.
I use q() or qq().
These are quote-like operators for single quote and double quote.
There are, of course, many more:
https://perldoc.perl.org/perlop#Quote-and-Quote-like-Operators
gizmo _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Thank you, I will look at those (the page says they should be q{} and q{} rather than q() and q()). It would still be useful for me to understand why my combination of single and double quotes in the perl command does not work? Just for "fun" I also tried concatenating the different strings in the perl command with . which I understand is the perl string concatenation symbol but again to no avail.
May I ask you what the correct way of including a literal $ in a substitution string would be? In bash I would precede it with a single \ but that led to loss of any text after that on that same line when used in the text string I use in my perl command, neither did preceding it with \ work.
I have seen that using an ENV() construct would work but I would prefer something simpler, if at all possible.
I have used literal # in my substitution string without any problems but have not gotten literal $ to work.
Thanks.
On 2/3/22 09:21, H wrote:
On 02/03/2022 08:39 AM, Joe Kline wrote:
I avoid using ' or " in one-liners or even programs.
I use q() or qq().
These are quote-like operators for single quote and double quote.
There are, of course, many more:
https://perldoc.perl.org/perlop#Quote-and-Quote-like-Operators
gizmo _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Thank you, I will look at those (the page says they should be q{} and q{} rather than q() and q()). It would still be useful for me to understand why my combination of single and double quotes in the perl command does not work? Just for "fun" I also tried concatenating the different strings in the perl command with . which I understand is the perl string concatenation symbol but again to no avail.
May I ask you what the correct way of including a literal $ in a substitution string would be? In bash I would precede it with a single \ but that led to loss of any text after that on that same line when used in the text string I use in my perl command, neither did preceding it with \ work.
I have seen that using an ENV() construct would work but I would prefer something simpler, if at all possible.
I have used literal # in my substitution string without any problems but have not gotten literal $ to work.
I think it's been pointed out that it might be a lot easier to stick with either perl or shell.
For shell you could probably do:
( cat - ; echo $txt ) >output
As for perl, I guess I'll leave that as an exercise for you how to translate from shell to perl.
For the quote-like operators you can use either () or {} for the delimiter for almost all of the quote-like operators, in fact, from the doc:
"In the following table, a {} represents any pair of delimiters you choose."
gizmo
On Wed, Feb 02, 2022 at 08:54:38PM -0500, H wrote:
I am writing a long bash script under CentOS 7 where perl is used for manipulating some external files. So far I am using perl one-liners to do so but ran into a problem when I need to append text to an external file.
Here is a simplified example in the bash script where txt is a bash variable which I built containing a longish text with multiple newlines:
txt="a b$'\n'cd ef$'\n'g h$'\n'ij kl"
A simplified perl one-liner to append the text in the variable above to some file in the bash script would be:
perl -pe 'eof && do{print $_'"${txt}"'; exit}' someexternalfile.txt
This works when fine when $txt does /not/ contain any spaces but falls apart when it does.
In a shell script why not stick to shell tools?
printf "%s" "${txt}" >> someexternalfile.txt
On 02/02/2022 11:34 PM, Jon LaBadie wrote:
On Wed, Feb 02, 2022 at 08:54:38PM -0500, H wrote:
I am writing a long bash script under CentOS 7 where perl is used for manipulating some external files. So far I am using perl one-liners to do so but ran into a problem when I need to append text to an external file.
Here is a simplified example in the bash script where txt is a bash variable which I built containing a longish text with multiple newlines:
txt="a b$'\n'cd ef$'\n'g h$'\n'ij kl"
A simplified perl one-liner to append the text in the variable above to some file in the bash script would be:
perl -pe 'eof && do{print $_'"${txt}"'; exit}' someexternalfile.txt
This works when fine when $txt does /not/ contain any spaces but falls apart when it does.
In a shell script why not stick to shell tools?
printf "%s" "${txt}" >> someexternalfile.txt
I want to use similar patterns of perl one-liners for more complicated text substitutions. If I cannot get the simple example above to work, I surely cannot get more complicated text substitutions, including substitutions spanning multiple lines, to work.
On 1/30/22 17:12, H wrote:
I am writing a long bash script under CentOS 7 where perl is used for manipulating some external files. So far I am using perl one-liners to do so but ran into a problem when I need to append text to an external file.
Here is a simplified example in the bash script where txt is a bash variable which I built containing a longish text with multiple newlines:
txt="a b$'\n'cd ef$'\n'g h$'\n'ij kl"
A simplified perl one-liner to append the text in the variable above to some file in the bash script would be:
perl -pe 'eof && do{print $_'"${txt}"'; exit}' someexternalfile.txt
This works when fine when $txt does /not/ contain any spaces but falls apart when it does.
I would like to keep the above structure, ie using bash variables to build text strings and one-liners to do the text manipulation. Hopefully there is a "simple" solution to do this, I have tried many variations and failed miserably... Note that I also want to use a similar pattern to do substitutions in external files, I would thus like to use the same code pattern.
Thanks.
On 2/2/22 17:54, H wrote:
I am still having a problem. The following (where $txt is an
arbitrary string) works:
perl -e 'print '""${txt}""';'
The following does not work (I want to append the content of the $txt
to the end of an existing file in-place):
perl -i -pe 'eof && do{print $_''"aaa"''; exit}' somefile.txt
but this does:
perl -i -pe "eof && do{print $_"""${txt}""'; exit}' somefile.txt
as does:
perl -i -pe "eof && do{print $_"""${txt}"""; exit}" somefile.txt
The difference is that the last two perl command strings use " rather
than '.
My questions are:
- Why would not using single-quotes for parts of the perl command
string work?
- Is there any reason I should fight this or should I just go with
double-quotes for all parts of the perl command string? Any downside? Remember, these are all in bash scripts and I am looking for a "pattern" to use for other, more complicated text substitutions, hence the use of perl.
On 2/2/22 18:55, H wrote:
I see I made a mistake, the line:
perl -i -pe 'eof && do{print $_''"aaa"''; exit}' somefile.txt
should be:
perl -i -pe 'eof && do{print $_''"${txt}"''; exit}' somefile.txt
Related question, if the $txt string contains eg $ or another special
character, what would be the best way of escaping it so it is not substituted by perl?
AIUI you are looking for a Bash shell scripting (programming) technique that allows you to append content to a file using a Perl one-liner with data that is dynamically generated from Bash variable values, all inside a Bash script (?).
Perhaps it would be simpler if you put the data into a file and then invoked the Perl one-liner with the data file filename as an argument:
2022-02-02 22:08:34 dpchrist@tinkywinky ~ $ cat somefile.txt this is the contents of somefile. some more contents.
2022-02-02 22:08:37 dpchrist@tinkywinky ~ $ cat centos-h.sh #!/bin/bash txt="... foo ..." echo "$txt" > tmp.txt perl -pe 's/foo/bar/g' tmp.txt >> somefile.txt
2022-02-02 22:08:46 dpchrist@tinkywinky ~ $ bash centos-h.sh
2022-02-02 22:08:53 dpchrist@tinkywinky ~ $ cat somefile.txt this is the contents of somefile. some more contents. ... bar ...
David
-
David Christensen -
H -
Joe Kline -
Jon LaBadie -
mailist -
Orion Poplawski