# Reading and writing files; line endings > Redirections, reading a file line by line, and the three things real files break: no newline after the last line, extra empty lines and Windows CRLF line endings – how to see them with cat -et and the one loop pattern that handles them all. Source: https://laszlonemes.com/university/computer-systems/files-line-endings · Author: László Nemes Session 5 · Computer Systems Redirections, reading a file line by line, and the three things real files break: no newline after the last line, extra empty lines and Windows CRLF line endings – how to see them with cat -et and the one loop pattern that handles them all. 20 min read `> vs >>` `while IFS= read -r line` `LF vs CRLF` `Last line without newline` `cat -et and od -c` `The universal loop pattern` Most shell tasks – in class, in the practical test and in your assignment – read a file, do something with every line and write the result somewhere. The loop itself is short. What breaks it is the file: a last line without a newline, a few empty lines at the end, or a file that was saved on Windows. All three are invisible in an editor and all three change what your script does. Here is the symptom in one line. A file holds a single word, `banana`, and `wc -l` says it has **0** lines. By the end of this session you will know why, how to see it, and the one loop pattern that handles all of these files correctly. After this session you can - redirect input and output with `<`, `>`, `>>` and know which one empties the file - read a file line by line with `while IFS= read -r line` and split lines into fields - see invisible characters with `cat -et` and `od -c` - explain and fix the three file traps: no final newline, extra empty lines, CRLF line endings - start every file-processing script from one safe loop pattern > **The whole session in one sentence:** a line is the text *before* a newline character, and Unix tools only count `\n` as a newline – so a file that ends without one, or uses `\r\n`, is not what your loop thinks it is. ## Reading and writing files ### Redirections A command reads standard input and writes standard output. Redirections point those streams at files instead of the terminal. You have met some of them already; these are the ones file processing needs. | Operator | Meaning | | --- | --- | | `cmd < file.txt` | Input comes from the file. | | `cmd > file.txt` | Output goes to the file. The old content is **deleted first** (a missing file is created). | | `cmd >> file.txt` | Output is **appended** to the end of the file; the old content stays. | | `> file.txt` | On its own: empty the file, or create an empty one. | | `cmd << EOF … EOF` | A multi-line block written in the script is the input (here document). | | `cmd <<< "text"` | One string or variable is the input (here string). | | `cmd >&2` | Output goes to standard error – use it for error messages. | > **Fig. 1** · A single `>` truncates before it writes; `>>` appends. > > *Diagram:* Three commands in a row. The single > empties the file before writing, so the second echo replaces apple with pear. The double >> appends, so plum is added after pear. ```shell echo "apple" > fruits.txt # fruits.txt: apple echo "pear" > fruits.txt # fruits.txt: pear (apple is gone) echo "plum" >> fruits.txt # fruits.txt: pear, plum (appended) ``` The two combine into the standard shape for writing a result file: empty it **once** before the loop, then append inside the loop. Without the first line, running the script twice would leave the first run's output in the file, with the second run's output after it. ```shell > "$output_file" # empty it once, before the loop while IFS= read -r line || [ -n "$line" ]; do echo "$line" >> "$output_file" # append inside the loop done < "$input_file" ``` Common mistake **Reading and writing the same file** `sort data.txt > data.txt` leaves you with an empty file. The shell sets up the redirection – and truncates `data.txt` – before `sort` even starts, so `sort` reads nothing. Write to a temporary file and `mv` it over the original. If the user passes the file names, refuse the same file twice; the `-ef` test is true when two names point to the same file. ```shell if [ "$input_file" -ef "$output_file" ]; then # -ef: same file, maybe under two names echo "Error: the input and the output file must be different" >&2 exit 1 fi ``` ### Line by line For real files, this is the loop header to use. Every part of it has a job: ```shell while IFS= read -r line || [ -n "$line" ]; do ... done < "$file" ``` | Part | What it does | | --- | --- | | `done < "$file"` | The whole loop reads from the file; each `read` takes the next line. | | `-r` | A backslash in the line stays a backslash instead of being treated as an escape. | | `IFS=` | Leading and trailing spaces are kept. Leave it out when you want them trimmed. | | `\|\| [ -n "$line" ]` | The last line is processed even if no newline follows it – see Trap 1. | When a line is made of fields, such as `Anna Kiss;87`, set `IFS` to the separator and give `read` one variable per field. If there are more fields than variables, the rest of the line goes into the last variable. ```shell while IFS=';' read -r name points || [ -n "$name" ]; do echo "$name: $points" done < grades.txt ``` Common mistake **for line in $(cat file) is not line by line** The `$(cat file)` output is split at every space, tab and newline, so `Anna Kiss` becomes two iterations, and empty lines disappear. For lines, always use `while read`. ```shell for line in $(cat names.txt); do # WRONG: "Anna Kiss" becomes "Anna" and "Kiss" echo "$line" done ``` Good to know **echo or printf?** `echo` always adds a newline after the text; `echo -n` and `printf '%s'` do not. That is exactly how a file without a final newline is made – keep it in mind when you write output files. ## What is a line? In a text file a line is ended by a **newline character**. Which character that is depends on the system: | System | Line ending | Written as | | --- | --- | --- | | Linux, macOS | LF (line feed) | `\n` | | Windows | CR LF (carriage return + line feed) | `\r\n` | The names come from typewriters: the carriage return pushes the carriage back to the start of the line, the line feed rolls the paper up by one line. Windows kept both; Unix systems kept only the line feed. Bash and the Unix tools – `read`, `grep`, `wc` – treat `\n` as the end of a line, and in a well-formed file **the last line ends with one too**. The four test files of this session (`numbers_*.txt`) contain the same three numbers – 10, 20, 30 – and still behave differently, because their bytes differ (Fig. 2). > **Fig. 2** · Same numbers, four different files. Only the first one is what a naive loop expects. > > *Diagram:* The four test files hold the same three numbers, but their bytes differ. The regular file ends every line, including the last, with a newline. One file has no newline after the last line, one has two extra empty lines at the end, and the Windows file has a carriage return before every newline. Plain `cat` doesn't show any of this. Use `cat -et`: it prints `$` at every newline, `^M` for a carriage return and `^I` for a tab. A last line without `$` has no newline. (On Linux `cat -A` is the same.) For the raw bytes, use `od -c`. ```shell cat -et file.txt # $ at every line end, ^M for \r, ^I for a tab od -c file.txt # every byte, \n and \r included tail -c 1 file.txt | od -c # only the last byte ``` cat -et on three of the files ```text $ cat -et numbers_ok.txt 10$ 20$ 30$ $ cat -et numbers_crlf.txt 10^M$ 20^M$ 30^M$ $ cat -et numbers_no_newline.txt 10$ 20$ 30$ ``` Good to know **The phantom empty line in your editor** Editors show one more empty line under the last line of a well-formed file. That is not an extra line, only the place where the cursor goes after the closing `\n`. An extra empty line exists only when two `\n` follow each other. Common mistake **Don't open the test files in an editor and save them** Many editors "fix" the end of a file on save – they add the missing newline or convert line endings. The trap you wanted to test is then gone. Look at test files with `cat -et`, don't edit them. ## Trap 1: no newline after the last line This is the most common one. Windows Notepad and several editors don't put a newline at the end of the file, and in a script `echo -n` or `printf '%s'` produce the same. Three things go wrong: | Symptom | Fix | | --- | --- | | The `while read` loop **skips the last line**: `read` does fill the variable, but returns failure because it found no newline. | `while IFS= read -r line \|\| [ -n "$line" ]` | | `wc -l` reports **one less**: it counts newline characters, not lines. | Count in the loop, or with `grep -c ''`. | | Appending with `>>` **glues the new line to the last one**. | Add the missing newline before appending. | Fig. 3 walks through the loop on the file `20\n30`. The second `read` gets `30` and returns failure in the same step. With a plain `while read` condition that failure ends the loop and `30` is lost; the `|| [ -n "$line" ]` part says "unless something was read". > **Fig. 3** · The `||` branch gives an unterminated last line one more pass through the body. > > *Diagram:* Reading a file whose last line has no newline. The second read gets 30 but returns failure because it hit the end of the file. The extra test, line is not empty, is true, so the loop body still runs for 30. The third read gets nothing and the loop ends. ```shell wc -l < numbers_no_newline.txt # 2 – counts \n characters grep -c '' numbers_no_newline.txt # 3 – counts lines ``` The third symptom destroys data, so it gets its own picture. A registry file stores one `name,phone` record per line; its last line has no newline; your script appends a new record. > **Fig. 4** · Appending to a file without a final newline merges two records into one broken line. > > *Diagram:* The registry file has no newline after its last line. Appending a record with >> glues it to that line, making one broken record. Adding the missing newline first, and only then appending, gives two proper lines. The fix checks the last byte first. `tail -c 1` prints it, and command substitution `$( )` strips a trailing newline from what it captures – so the result is empty exactly when the file ends in `\n`. ```shell # $( ) drops a trailing \n, so the result is empty exactly when the file ends in \n if [ -n "$(tail -c 1 "$file")" ]; then echo >> "$file" # add the missing newline first fi echo "$name,$phone" >> "$file" ``` ## Trap 2: extra empty lines `read` reads empty lines as well, so the loop runs more times than there are data lines. The count and the average come out wrong, a number check reports "invalid line", a copy script copies blank lines. The fix is to skip them: ```shell if [ -z "$line" ]; then continue fi ``` Do it **after** stripping the carriage return (next section): on a Windows file an "empty" line still contains `\r`, and `-z` would not see it as empty. ## Trap 3: Windows line endings (CRLF) When a file made on Windows is read on Linux, `read` cuts at `\n` and the `\r` stays at the end of the line. You can't see it in the terminal, but every comparison sees it (Fig. 5). > **Fig. 5** · The invisible `\r` rides along in the variable until you strip it. > > *Diagram:* A Windows line ends in carriage return plus newline. The read command cuts at the newline only, so the variable keeps the carriage return. The regex check fails, the string comparison fails and arithmetic errors out, until the carriage return is stripped. | What you do | What happens | | --- | --- | | `[[ $line =~ ^[0-9]+$ ]]` | False: `10\r` doesn't end in a digit. | | `[ "$line" == "end" ]` | False: `end\r` is not `end`. | | `sum=$(( sum + line ))` | Error: `invalid arithmetic operator`. | | `IFS=';' read -r name points` | The `\r` lands in the **last** field, `points`. | | The script itself has CRLF | `/bin/bash^M: bad interpreter` or `$'\r': command not found`. | what it looks like ```text $ ./sum.sh numbers_crlf.txt ./sum.sh: line 4: 10: syntax error: invalid arithmetic operator (error token is "\r") $ ./script.sh # the script itself was saved with CRLF -bash: ./script.sh: /bin/bash^M: bad interpreter: No such file or directory ``` **Where it comes from:** the file was created on Windows (Notepad, a CSV export from Excel), or git on Windows converted line endings on checkout (`core.autocrlf`). The course repository has a `.gitattributes` file that keeps `.sh` and `.txt` files LF everywhere – and leaves the deliberately broken test files alone. | Fix | Effect | | --- | --- | | `${line%$'\r'}` | Inside the loop: removes one `\r` from the end of the line. `$'\r'` is bash notation for the CR character. | | `tr -d '\r' < in.txt > out.txt` | Deletes every `\r` in the file. | | `tr -d '[:space:]'` | Deletes every whitespace character, `\r` included. Also deletes spaces *between* words – use it on numbers. | | `dos2unix file.txt` | The dedicated tool (in Git Bash; elsewhere it may need installing). | ```shell line=${line%$'\r'} # inside the loop: drop one \r from the end tr -d '\r' < in.txt > out.txt # the whole file at once dos2unix file.txt # the dedicated tool, where it is installed ``` Common mistake **Your script was saved with CRLF** You write the script on Windows, copy it to the server, run it – and get `/bin/bash^M: bad interpreter`. The first line is `#!/bin/bash\r`, and there is no program called `bash\r`. In VS Code, the status bar shows `CRLF` or `LF` at the bottom right; click it and choose `LF` before saving. On the server, strip it: ```shell sed -i 's/\r$//' myscript.sh # strip the \r from every line, in place cat -et myscript.sh | head -3 # check: no ^M left ``` Git Bash on Windows is more forgiving with CRLF scripts than bash on Linux. "It runs on my machine" doesn't mean it runs on the test machine – always try it on the server. ### Try it: the naive loop on all four files The loop you would write first, without any of the fixes: ```shell count=0; sum=0 while read -r number; do count=$(( count + 1 )) sum=$(( sum + number )) done < "$file" ``` | File | Naive: lines | Naive: sum | Safe pattern: sum | | --- | --- | --- | --- | | `numbers_ok.txt` | 3 | 60 | 60 | | `numbers_no_newline.txt` | **2** | **30** | 60 | | `numbers_empty_lines.txt` | **5** | 60 | 60 | | `numbers_crlf.txt` | 3 | **error** | 60 | The empty-lines file sums correctly only by luck: an empty variable counts as 0 in arithmetic. The count – and any average – is still wrong. ## The universal pattern Put the three fixes together and you get the loop to start every file-processing script from. Copy it, and only rewrite the "process" part. read a file line by line – the safe way ```shell while IFS= read -r line || [ -n "$line" ]; do line=${line%$'\r'} # 1. drop a Windows \r if [ -z "$line" ]; then # 2. skip empty lines continue fi # 3. --- process the line --- echo "$line" done < "$file" ``` > **Fig. 6** · Every line passes the same three filters before your code sees it. > > *Diagram:* The universal line-reading loop as a chain: the file feeds the loop, read keeps even an unterminated last line, the carriage return is stripped, empty lines are skipped, and only clean, non-empty lines reach your processing code. | Part | What it solves | | --- | --- | | `IFS= read -r line` | The line lands in the variable exactly as in the file (spaces, backslashes). | | `\|\| [ -n "$line" ]` | The last line is kept even without a final newline. | | `${line%$'\r'}` | The trailing `\r` of a Windows file is removed. | | `[ -z "$line" ]` + `continue` | Empty lines, including those at the end, are skipped. | | `done < "$file"` | The loop reads the file. Unlike `cat file \| while …`, a counter or array changed in the loop keeps its value after the loop – a pipe would run the loop in a subshell. | Three common variations: - **Fields:** `while IFS=';' read -r name points || [ -n "$name" ]`. The `\r` is now in the **last** field, so strip it there: `points=${points%$'\r'}`. - **Numbers:** `number=$(echo "$line" | tr -d '[:space:]')` removes spaces, tabs and the `\r` in one step. - **Output file:** `> "$output_file"` before the loop, `echo "…" >> "$output_file"` inside. And the checks that come before the loop in every task of this session: ```shell if [ "$#" -ne 1 ]; then echo "Usage: $0 " >&2 exit 1 fi if [ ! -f "$1" ]; then echo "Error: file '$1' does not exist" >&2 exit 1 fi ``` In your assignment **Test your script on bad files, not only the good one** The assignment is checked on the Linux server, often with input you didn't write. Start every file-reading part from the universal pattern, then run it on at least three inputs: a normal file, a file without a final newline (`printf '10\n20'`) and a CRLF file (`printf '10\r\n20\r\n'`). If the three results differ, the script isn't finished. Error messages go to standard error with `>&2` and the script stops with `exit 1`; a requested `--help` goes to standard output with `exit 0`. - **LF** – `\n`, line feed – the line ending on Linux and macOS. - **CR** – `\r`, carriage return – on Windows it comes before every LF. - **CRLF** – `\r\n`, the Windows line ending. - **truncate** – Empty a file to 0 bytes – what `>` does before writing. - **here string** – `<<< "$var"` – one string as a command's input. ## Practice For every task: check the number of arguments and that the input file exists, write error messages to standard error and stop with `exit 1`. Tasks 1–5 are general scripting warm-ups; 6–9 read and write files and include the traps on purpose. Run your script on every `numbers_*.txt` file, and compare with `cat -et`. Task 1 **Factorial** Compute n! for a number given as an argument. Refuse numbers above 20. ```text $ ./factorial.sh 5 5! = 120 $ ./factorial.sh 21 Error: the number must be at most 20 ``` **Approach:** Start the product at 1, so 0! is right without a special case. Bash computes with 64-bit integers: 21! no longer fits and silently overflows – that is why the limit is 20. ```shell result=1 # 0! = 1 falls out for free for (( i=2; i<=n; i++ )); do result=$(( result * i )) done ``` Task 2 **Sorted words** Read words until the word `end` is typed, then print them sorted. It must also work when the input comes from a file (`./sorted_words.sh < words.txt`) that has no `end` and no newline after its last word. **Approach:** Collect the words in an array, stop on `end` with `break` – the loop also stops at the end of the file. Thanks to `|| [ -n "$word" ]` the unterminated last word isn't lost. ```shell while read -r word || [ -n "$word" ]; do [ "$word" == "end" ] && break words+=("$word") done printf '%s\n' "${words[@]}" | sort ``` Task 3 **Add up the arguments** Add up at most 10 integers given as arguments and print the sum. With `--help` instead of numbers, print a short usage text. ```text $ ./sum.sh 4 -2 10 Sum: 12 $ ./sum.sh 1 x Error: 'x' is not an integer ``` **Approach:** Check `--help` **before** the argument count. Help that was asked for is not an error: standard output and `exit 0`. Then check `"$#"` is between 1 and 10. ```shell if [ "$1" == "--help" ]; then # before the argument count check echo "Usage: $0 [n2 ... n10]" exit 0 # help that was asked for is not an error fi for n in "$@"; do [[ $n =~ ^-?[0-9]+$ ]] || { echo "Error: '$n' is not an integer" >&2; exit 1; } sum=$(( sum + n )) done ``` Task 4 **Odd and even lines** Split the file given as an argument into `odd_lines.txt` and `even_lines.txt` by line number. The test file's last line has no newline – it must still end up in `odd_lines.txt`. **Approach:** Count lines yourself; empty both outputs once before the loop. ```shell > odd_lines.txt; > even_lines.txt while IFS= read -r line || [ -n "$line" ]; do n=$(( n + 1 )) if (( n % 2 == 1 )); then echo "$line" >> odd_lines.txt else echo "$line" >> even_lines.txt; fi done < "$file" ``` Task 5 **Phone registry** A registry file stores `Name,number` lines. Write a script with options: `-a ` adds an entry (no duplicate names), `-d ` deletes one, `-s ` searches names case-insensitively and prints their numbers, `-c` prints the number of entries, `-o` lists the names sorted, each once. The file's last line has no newline. ```text $ ./registry.sh -a "Dan Szabo" 06301112222 Added: Dan Szabo,06301112222 $ ./registry.sh -s szab Dan Szabo: 06301112222 ``` **Approach:** A `case` on `$1` calls one function per option. `^$name,` matches the name exactly at the start of the line. Before `-a` appends, add the missing newline (Fig. 4). Count with `grep -c ''`, not `wc -l`. ```shell grep -q "^$name," "$registry" && { echo "Error: '$name' exists" >&2; exit 1; } grep -v "^$name," "$registry" > "$registry.tmp" # delete = keep everything else mv "$registry.tmp" "$registry" grep -c '' "$registry" # count, not wc -l cut -d ',' -f 1 "$registry" | sort -u # names, sorted, once each ``` Task 6 **What is at the end of the file?** For a file given as an argument, print: the line count according to `wc -l` and according to a `read` loop, whether it ends with a newline, how many empty lines it has, and how many lines end in `\r\n`. Run it on all four `numbers_*.txt` files. ```text File: numbers_no_newline.txt Lines (wc -l): 2 Lines (read loop): 3 Ends with newline: NO Empty lines: 0 Lines ending in CRLF: 0 ``` **Approach:** Count CRLF lines **before** stripping the `\r`; count empty lines after. The final-newline test is the `tail -c 1` trick. ```shell if [[ $line == *$'\r' ]]; then # the line still ends in \r crlf=$(( crlf + 1 )) fi [ -z "$(tail -c 1 "$file")" ] && ends="yes" || ends="NO" ``` Task 7 **Sum numbers from file to file** Read one integer per line from an input file. Copy the valid numbers to an output file and append `Sum: ` at the end. Skip empty lines; for invalid lines print a warning with the line number. It must give the right result on every `numbers_*.txt` file, including `numbers_mixed.txt`, which has every trap at once – spaces, tabs, CRLF, empty and invalid lines. ```text $ ./sum_numbers.sh numbers_mixed.txt result.txt Warning: line 6 is not an integer, skipped: 'abc' Warning: line 8 is not an integer, skipped: '1.5' Sum: 133 ``` **Approach:** Refuse the same file as input and output (`-ef`). Clean each line with `tr -d '[:space:]'` – that handles spaces, tabs and `\r` together – then validate with a regex. Keep your own line counter for the warnings. ```shell number=$(echo "$line" | tr -d '[:space:]') # spaces, tabs and \r in one go [ -z "$number" ] && continue if [[ ! $number =~ ^-?[0-9]+$ ]]; then echo "Warning: line $line_number is not an integer, skipped: '$number'" >&2 continue fi ``` Task 8 **Clean up a text file** From an input file, produce a clean output file: no `\r`, no spaces at the start or end of lines, no empty lines, and a newline after the last line. Check the result with `cat -et`. ```text $ cat -et clean.txt First line$ Second line, had leading spaces$ Fourth line, had an empty line before$ Fifth line$ ``` **Approach:** Delete the `\r` with `tr` first. Then use `read -r line` **without** `IFS=` – this time trimming is what you want. `echo` writes a newline after every line, the last one included. One redirection after `done` writes the whole loop's output. ```shell tr -d '\r' < "$input_file" | while read -r line || [ -n "$line" ]; do [ -n "$line" ] && echo "$line" # read without IFS= trims both ends done > "$output_file" ``` Task 9 **Grades report** `grades.txt` starts with a header line, then `name;points` lines – with CRLF line endings, empty lines, an invalid score and no newline after the last line. Write a report file with each student's name, points and grade, then the number of students, the average and the best student. Grade limits: 0–49: 1, 50–59: 2, 60–69: 3, 70–84: 4, 85–100: 5. ```text name;points;grade Anna Kiss;87;5 Bela Nagy;45;1 Carol Toth;92;5 … --- Students: 7 Average: 64 Best: Carol Toth (92) ``` **Approach:** The `\r` ends up in `points`, the last field – without cleaning it, no score is a number. Grade with an `if/elif` chain from the top down (`-ge 85`, `-ge 70` …). Before dividing for the average, check the count isn't 0. ```shell while IFS=';' read -r name points || [ -n "$name" ]; do line_number=$(( line_number + 1 )) [ "$line_number" -eq 1 ] && continue # header points=$(echo "$points" | tr -d '[:space:]') # the \r sits in the LAST field ... done < "$input_file" ``` ## More practice Every task reads from a file and writes to a file, and the input files contain this session's traps. Recreate the output file on every run – running twice must give the same result. Task 10 **Number the lines** Write the lines of `poem.txt` to a new file as `: `. Empty lines stay empty and get no number. The last line has no newline; it must still appear, numbered 5. **Approach:** Here an empty line is not skipped with `continue` alone – write an empty line first. ```shell if [ -z "$line" ]; then echo >> "$output_file" # keep the empty line, without a number continue fi number=$(( number + 1 )) echo "$number: $line" >> "$output_file" ``` Task 11 **Inventory value** `inventory.txt` has a header line, then `product;quantity;unit price` lines (CRLF, empty lines). Write each product and its value (quantity × price) to `inventory_report.txt`, then the total. Invalid lines go to `errors.txt` with their line number. Expected: total 18840, 2 invalid lines. **Approach:** Split with `IFS=';'`, skip line 1, clean both numbers, validate both. ```shell while IFS=';' read -r product quantity price || [ -n "$product" ]; do quantity=$(echo "$quantity" | tr -d '[:space:]') price=$(echo "$price" | tr -d '[:space:]') if [[ ! $quantity =~ ^[0-9]+$ ]] || [[ ! $price =~ ^[0-9]+$ ]]; then echo "line $line_number: $product" >> errors.txt continue fi ``` Task 12 **Split a log file** Lines of `server.log` look like `date time LEVEL message`. Sort them into `info.log`, `warning.log` and `error.log`, and print how many lines each level had. Running it twice must give the same files. Expected: 5 INFO, 2 WARNING, 2 ERROR – the last line has no newline. **Approach:** Empty all three outputs before the loop. A here string splits the line at spaces; the message, with its spaces, goes into the last variable. ```shell read -r date time level message <<< "$line" case "$level" in INFO) echo "$line" >> info.log ;; WARNING) echo "$line" >> warning.log ;; ERROR) echo "$line" >> error.log ;; esac ``` Task 13 **Merge two files** Line i of `names.txt` belongs to line i of `emails.txt`. Write them to `contacts.txt` as `name;email`. If one file is shorter, the missing field stays empty (`Emese Szabo;`). **Approach:** Read each file into its own array (with the universal pattern), then loop with an index up to the longer length. A missing array element expands to an empty string. ```shell count=${#names[@]} [ "${#emails[@]}" -gt "$count" ] && count=${#emails[@]} for (( i=0; i> "$output_file" # a missing element is just "" done ``` Task 14 **Extend the phone registry** Add two options to task 5: `-e ` changes a number, `-i ` imports entries from a file. On import, trim spaces and `\r`, skip invalid lines and names already in the registry. Expected with `new_contacts.txt`: 2 new names, 1 already present, 1 invalid. **Approach:** Editing = rewrite every line to a temporary file, the matching one changed, then `mv`. During import, a bad line is `continue`, not `exit` – one bad line must not stop the whole import. ```shell while IFS=, read -r current_name current_phone || [ -n "$current_name" ]; do if [ "$current_name" == "$name" ]; then echo "$name,$phone" else echo "$current_name,$current_phone"; fi done < "$registry" > "$registry.tmp" mv "$registry.tmp" "$registry" ``` ## Common mistakes | Wrong | Right | Why | | --- | --- | --- | | `while read -r line` | `while IFS= read -r line \|\| [ -n "$line" ]` | Loses an unterminated last line. | | `for line in $(cat file)` | `while IFS= read -r line …; done < file` | `for` splits into words, not lines. | | `wc -l` as the line count | `grep -c '' file` or count in the loop | `wc -l` counts newline characters. | | `echo "$new" >> file` | first `[ -n "$(tail -c 1 file)" ] && echo >> file` | The new line can glue onto the last one. | | regex on a Windows file | first `line=${line%$'\r'}` | The invisible `\r` is at the end of the line. | | `>>` in the loop, no emptying | `> "$out"` before the loop | The second run appends to the first. | | `cmd < f.txt > f.txt` | temporary file, then `mv` | `>` empties the file before it is read. | | `avg=$(( sum / count ))` | first `if [ "$count" -eq 0 ]` | An empty file divides by zero. | ## Summary - `>` empties then writes, `>>` appends. Empty the output once before the loop, append inside it. - A line ends with `\n`. Windows adds a `\r` before it; some files have no `\n` after the last line. - `cat -et` shows the truth: `$` = `\n`, `^M` = `\r`, no `$` at the end = no final newline. - Start every file loop from the universal pattern: `|| [ -n "$line" ]`, strip `\r`, skip empty lines. - Test on a normal file, an unterminated one and a CRLF one – on the Linux server, not only in Git Bash.