Stripping ^M from your files

Stripping ^M from your files

If you have transferred a file from MS Windows to FreeBSD, you might find
the file looks like this:

<html>^M
^M
<head>^M
<title>The FreeBSD Diary -- Article Maintenance</title>^M
<meta name="description" content="BLANK">^M
<meta name="keywords" content="FreeBSD">^M

That "^M" is control-M, which is a carriage return, and is not
needed in Unix file systems.  To remove this extra character, run the following
script:

perl -pi -e "s:^V^M::g" <filenames>

where ^V is actually control-V and ^M is actually control-M (you must type these
yourself, don’t just copy and paste this command).  Control-V will not be displayed
on your screen.

<filenames> is the list of files to be converted.

NOTE: I’m told that if you use ASCII mode to transfer your files, this problem won’t
occur.

Other options

I like people writing in with options.  Here are two other ways to
get rid of the ^M characters:

  • Marc Silver writes that this will work:
cat <filename1> | tr -d "^V^M" > <newfile>    

Remember to actually type control-V then control-M.  Don’t just copy and paste the
above.

  • Trevor R.H. Clarke gave this example:
sed -e "s/^V^M//" <filename> > <output filename>

And remember to actually type control-V then control-M.  Don’t just copy and paste
the above.

  • Mario Sergio Fujikawa Ferreira writes of this vi solution.
  1. hit the ESC key
  2. :%s/^V^M//

remember the colon ( : ) at the start of step two.

  • Marius Strom mentions this port:
    cd /usr/ports/converters/unix2dos make && make install

    This port will also install dos2unix, which is useful for removing the
    control-M characters.

  • Oliver Crow reports this:
    col <infile >outfile

    [ed. note: make sure infile and outfile are not the same file or
    you’ll lose the contents]

  • KeMpKeS reports this:

    Here’s one you might want to add to the removing ^M from files page:

    open the file in pico, make a trivial change, then change it back (like
    add a space then delete it), then try to quit pico. When it asks you if
    you want to save changes, say yes. That”ll do it.

8 thoughts on “Stripping ^M from your files”

  1. #!/usr/local/bin/bash
    echo "Removing ^M’s for the following files…"
    for file in $*; do
    echo "—-> $file"
    tr -d "\015" < $file > $file.$$
    mv $file.$$ $file
    done
    echo "Done."
    exit 0

    1. for those that have access to the rename command:
      rename .jpg^M .jpg *.jpg^M

      that is entered in the directory containing the offending files like:
      rename SPACE .jpg CNTRL-V CNTRL-M SPACE .jpg SPACE *.jpg CNTRL-V CNTRL-M ENTER.

      CNTRL-V CNTRL-M is required to get the actual CNTRL-M into the command line.
      the .jpg is an example extention, but could really be anything… .html, .txt, etc.

      That’ll take off the ^M from all the file names.

    1. In VIM (both windows and UNIX), the preferred approach is
      to use \r instead of ^V^M.

      :%s/\r//

      will do it. I assume you can use "\r" similarly in perl but
      have not tested it.

      [%sig%]

  2. I found this today when I logged in. Fortune told me:

    tr -d \\r < dosfile > newfile
    — Originally by Dru <genesis@istar.ca>

    This is similar to what Marc Silver mentioned, as shown in the original article.


    The Man Behind The Curtain

  3. After a long painful learning curve I came up with this solution to strip out the ^M characters Windows leaves.

    sed ‘s/\^M$//’ file > tfile
    mv tfile file

    the ^M (ascii 13) has to be entered as the actual ascii character. The $ is what sed uses to indicate the end of line. This works under sh.

    This script only strips the ^M’s off of the end of the lines.

    [%sig%]

    Post Edited (21-09-11 15:39)

Leave a Comment

Scroll to Top