Stripping ^M from your files
If you have transferred a file from MS Windows to FreeBSD, you might find the file looks like this:<html>^M ^M <head>^M <title>The FreeBSD Diary -- Article Maintenance</title>^M <meta name="description" content="BLANK">^M <meta name="keywords" content="FreeBSD">^M
That "^M" is control-M, which is a carriage return, and is not needed in Unix file systems. To remove this extra character, run the following script:
perl -pi -e "s:^V^M::g" <filenames>
where ^V is actually control-V and ^M is actually control-M (you must type these yourself, don’t just copy and paste this command). Control-V will not be displayed on your screen.
<filenames> is the list of files to be converted.
NOTE: I’m told that if you use ASCII mode to transfer your files, this problem won’t occur.
Other options
I like people writing in with options. Here are two other ways to get rid of the ^M characters:- Marc Silver writes that this will work:
cat <filename1> | tr -d "^V^M" > <newfile>
Remember to actually type control-V then control-M. Don’t just copy and paste the above.
- Trevor R.H. Clarke gave this example:
sed -e "s/^V^M//" <filename> > <output filename>
And remember to actually type control-V then control-M. Don’t just copy and paste the above.
- Mario Sergio Fujikawa Ferreira writes of this vi solution.
- hit the ESC key
- :%s/^V^M//
remember the colon ( : ) at the start of step two.
- Marius Strom mentions this port:
cd /usr/ports/converters/unix2dos make && make install
This port will also install dos2unix, which is useful for removing the control-M characters.
- Oliver Crow reports this:
col <infile >outfile
[ed. note: make sure infile and outfile are not the same file or you’ll lose the contents]
- KeMpKeS reports this:
Here’s one you might want to add to the removing ^M from files page: open the file in pico, make a trivial change, then change it back (like add a space then delete it), then try to quit pico. When it asks you if you want to save changes, say yes. That”ll do it.
You should also read <A HREF="/phorum/read.php?f=1&i=2740&t=2740">this thread</A> in the <A HREF="/phorum/list.php?f=1">Reader Forum</A>.
$strings filename >newfilename
#!/usr/local/bin/bash
echo "Removing ^M’s for the following files…"
for file in $*; do
echo "—-> $file"
tr -d "\015" < $file > $file.$$
mv $file.$$ $file
done
echo "Done."
exit 0
for those that have access to the rename command:
rename .jpg^M .jpg *.jpg^M
that is entered in the directory containing the offending files like:
rename SPACE .jpg CNTRL-V CNTRL-M SPACE .jpg SPACE *.jpg CNTRL-V CNTRL-M ENTER.
CNTRL-V CNTRL-M is required to get the actual CNTRL-M into the command line.
the .jpg is an example extention, but could really be anything… .html, .txt, etc.
That’ll take off the ^M from all the file names.
See also <a href="http://www.freebsddiary.org/phorum/read.php?f=4&i=125&t=125">http://www.freebsddiary.org/phorum/read.php?f=4&i=125&t=125</a>.
In VIM (both windows and UNIX), the preferred approach is
to use \r instead of ^V^M.
:%s/\r//
will do it. I assume you can use "\r" similarly in perl but
have not tested it.
[%sig%]
I found this today when I logged in. Fortune told me:
tr -d \\r < dosfile > newfile
— Originally by Dru <genesis@istar.ca>
This is similar to what Marc Silver mentioned, as shown in the original article.
—
The Man Behind The Curtain
After a long painful learning curve I came up with this solution to strip out the ^M characters Windows leaves.
sed ‘s/\^M$//’ file > tfile
mv tfile file
the ^M (ascii 13) has to be entered as the actual ascii character. The $ is what sed uses to indicate the end of line. This works under sh.
This script only strips the ^M’s off of the end of the lines.
[%sig%]
Post Edited (21-09-11 15:39)