Stripping ^M from your filesIf you have transferred a file from MS Windows to FreeBSD, you might find the file looks like this:
<html>^M ^M <head>^M <title>The FreeBSD Diary -- Article Maintenance</title>^M <meta name="description" content="BLANK">^M <meta name="keywords" content="FreeBSD">^M
That "^M" is control-M, which is a carriage return, and is not needed in Unix file systems. To remove this extra character, run the following script:
perl -pi -e "s:^V^M::g" <filenames>
where ^V is actually control-V and ^M is actually control-M (you must type these yourself, don’t just copy and paste this command). Control-V will not be displayed on your screen.
<filenames> is the list of files to be converted.
NOTE: I’m told that if you use ASCII mode to transfer your files, this problem won’t occur.
Other optionsI like people writing in with options. Here are two other ways to get rid of the ^M characters:
- Marc Silver writes that this will work:
cat <filename1> | tr -d "^V^M" > <newfile>
Remember to actually type control-V then control-M. Don’t just copy and paste the above.
- Trevor R.H. Clarke gave this example:
sed -e "s/^V^M//" <filename> > <output filename>
And remember to actually type control-V then control-M. Don’t just copy and paste the above.
- Mario Sergio Fujikawa Ferreira writes of this vi solution.
- hit the ESC key
remember the colon ( : ) at the start of step two.
- Marius Strom mentions this port:
cd /usr/ports/converters/unix2dos make && make install
This port will also install dos2unix, which is useful for removing the control-M characters.
- Oliver Crow reports this:
col <infile >outfile
[ed. note: make sure infile and outfile are not the same file or you’ll lose the contents]
- KeMpKeS reports this:
Here’s one you might want to add to the removing ^M from files page: open the file in pico, make a trivial change, then change it back (like add a space then delete it), then try to quit pico. When it asks you if you want to save changes, say yes. That”ll do it.
You should also read <A HREF="/phorum/read.php?f=1&i=2740&t=2740">this thread</A> in the <A HREF="/phorum/list.php?f=1">Reader Forum</A>.
$strings filename >newfilename
echo "Removing ^M’s for the following files…"
for file in $*; do
echo "—-> $file"
tr -d "\015" < $file > $file.$$
mv $file.$$ $file
for those that have access to the rename command:
rename .jpg^M .jpg *.jpg^M
that is entered in the directory containing the offending files like:
rename SPACE .jpg CNTRL-V CNTRL-M SPACE .jpg SPACE *.jpg CNTRL-V CNTRL-M ENTER.
CNTRL-V CNTRL-M is required to get the actual CNTRL-M into the command line.
the .jpg is an example extention, but could really be anything… .html, .txt, etc.
That’ll take off the ^M from all the file names.
See also <a href="http://www.freebsddiary.org/phorum/read.php?f=4&i=125&t=125">http://www.freebsddiary.org/phorum/read.php?f=4&i=125&t=125</a>.
In VIM (both windows and UNIX), the preferred approach is
to use \r instead of ^V^M.
will do it. I assume you can use "\r" similarly in perl but
have not tested it.
I found this today when I logged in. Fortune told me:
tr -d \\r < dosfile > newfile
— Originally by Dru <email@example.com>
This is similar to what Marc Silver mentioned, as shown in the original article.
The Man Behind The Curtain
After a long painful learning curve I came up with this solution to strip out the ^M characters Windows leaves.
sed ‘s/\^M$//’ file > tfile
mv tfile file
the ^M (ascii 13) has to be entered as the actual ascii character. The $ is what sed uses to indicate the end of line. This works under sh.
This script only strips the ^M’s off of the end of the lines.
Post Edited (21-09-11 15:39)