I/O and streams

The principal I/O facilities in C# are the Stream hierarchy (FileStream, MemoryStream, NetworkStream, etc.), the reader and writer wrappers (StreamReader, StreamWriter), and the high-level convenience APIs on File and Directory. Asynchronous I/O is well-supported throughout: every reading method has an asynchronous counterpart that returns a Task. C# admits the using statement for guaranteed disposal of streams, which is essential because streams hold unmanaged resources that the GC will not promptly release.

The model is broadly similar to other modern languages — a stream abstraction, encoding-aware text wrappers, async support — with the .NET-specific addition of LINQ-friendly enumerable APIs and the Span<T>-based zero-allocation overloads that have been added in recent revisions.

The Stream hierarchy

The abstract Stream class is the base for byte-oriented I/O:

Stream
├─ FileStream         — file
├─ MemoryStream       — in-memory buffer
├─ NetworkStream      — socket
├─ BufferedStream     — wraps another stream with buffering
├─ CryptoStream       — wraps another stream with encryption
├─ GZipStream, DeflateStream  — compression
└─ ...

The principal operations on Stream:

• Read(buffer, offset, count) — read up to count bytes; returns the number actually read.
• Write(buffer, offset, count) — write count bytes.
• Seek(offset, origin) — reposition (if the stream supports seeking).
• Length, Position — for seekable streams.
• Flush() — ensure buffered writes reach the underlying medium.
• CanRead, CanWrite, CanSeek — capability queries.

Plus async counterparts:

• ReadAsync(buffer, offset, count, cancellationToken)
• WriteAsync(buffer, offset, count, cancellationToken)
• FlushAsync(cancellationToken)

using var stream = new FileStream(path, FileMode.Open);
byte[] buffer = new byte[4096];
int read = await stream.ReadAsync(buffer);

The streams are dispose-bound: a stream holds a file handle, a socket, or other resources that the runtime needs explicit release for. The conventional using statement (or using declaration) ensures the resource is released:

using (var stream = new FileStream(path, FileMode.Open)) {
    // use stream
}    // Dispose() called here

// or, since C# 8:
using var stream = new FileStream(path, FileMode.Open);
// ...
// Dispose() called at end of enclosing scope

FileStream

FileStream is the principal file-stream class:

using var input  = new FileStream(in_path,  FileMode.Open,    FileAccess.Read);
using var output = new FileStream(out_path, FileMode.Create,  FileAccess.Write);

byte[] buffer = new byte[4096];
int    read;
while ((read = await input.ReadAsync(buffer)) > 0) {
    await output.WriteAsync(buffer.AsMemory(0, read));
}

The constructor takes:

• path — the file path.
• FileMode — Open, Create, CreateNew, Append, Truncate, OpenOrCreate.
• FileAccess — Read, Write, ReadWrite.
• FileShare — controls how other processes may simultaneously open the file.
• bufferSize — internal buffer size (default 4096).
• useAsync — enables the OS’s async I/O.

The File.OpenRead, File.OpenWrite, and File.Create static methods are convenience constructors:

using var input = File.OpenRead(path);

MemoryStream

MemoryStream wraps a byte buffer:

using var ms = new MemoryStream();
ms.Write(data, 0, data.Length);
ms.Position = 0;

byte[] read = new byte[ms.Length];
ms.Read(read, 0, read.Length);

The principal use is in-memory buffering for tests, for serialisation, and for cases where a Stream interface is needed but the data is in memory rather than on disk.

MemoryStream.ToArray() produces a copy of the buffer; MemoryStream.GetBuffer() returns the internal buffer (potentially with capacity beyond the written length).

Readers and writers

The reader and writer wrappers convert between bytes and higher-level types:

Class	Purpose
StreamReader	Read text from a stream with character encoding
StreamWriter	Write text to a stream with character encoding
BinaryReader	Read primitive types (int, double, string) from a stream
BinaryWriter	Write primitive types to a stream
TextReader, TextWriter	Abstract bases for character I/O

StreamReader and StreamWriter

using var reader = new StreamReader(path, Encoding.UTF8);
string? line;
while ((line = await reader.ReadLineAsync()) != null) {
    ProcessLine(line);
}

using var writer = new StreamWriter(out_path, append: false, Encoding.UTF8);
await writer.WriteLineAsync("hello, world");
await writer.WriteLineAsync($"timestamp: {DateTime.Now}");

The reader and writer take an explicit Encoding; Encoding.UTF8 is the conventional contemporary choice. Without an encoding, the default is the system’s preferred encoding (typically UTF-8 on .NET 5+).

BinaryReader and BinaryWriter

using var writer = new BinaryWriter(stream);
writer.Write(42);                    // 4 bytes (int32)
writer.Write(3.14);                  // 8 bytes (double)
writer.Write("hello");               // length-prefixed UTF-8

using var reader = new BinaryReader(stream);
int    n = reader.ReadInt32();
double d = reader.ReadDouble();
string s = reader.ReadString();

The format is .NET-specific (length-prefixed strings, little-endian integers); two .NET programs that use the same BinaryReader/BinaryWriter interoperate cleanly. The format is not portable to other languages without explicit decoding.

The high-level File class

System.IO.File provides static convenience methods:

// Read entire file:
string text = File.ReadAllText(path);
string text2 = await File.ReadAllTextAsync(path);

byte[] bytes = File.ReadAllBytes(path);
string[] lines = File.ReadAllLines(path);

// Write entire file:
File.WriteAllText(path, contents);
await File.WriteAllTextAsync(path, contents);
File.WriteAllBytes(path, bytes);
File.WriteAllLines(path, lines);

// Append:
File.AppendAllText(path, "additional content\n");

// Existence and metadata:
bool exists = File.Exists(path);
DateTime modified = File.GetLastWriteTime(path);
long size = new FileInfo(path).Length;

// Manipulation:
File.Copy(source, destination);
File.Move(source, destination);
File.Delete(path);

// Streaming line iteration:
foreach (var line in File.ReadLines(path)) {
    ProcessLine(line);
}

File.ReadLines (singular) is deferred: it reads lines lazily as the enumerable is iterated. File.ReadAllLines (plural) reads all lines into a string array at once. The deferred form is appropriate for large files; the eager form is appropriate when the count of lines is small or when the array is needed.

The Directory class

using System.IO;

bool exists = Directory.Exists(path);
Directory.CreateDirectory(path);

string[] files = Directory.GetFiles(path);
string[] subdirs = Directory.GetDirectories(path);

foreach (string file in Directory.EnumerateFiles(path, "*.txt", SearchOption.AllDirectories)) {
    Console.WriteLine(file);
}

Directory.Delete(path, recursive: true);

Directory.GetFiles returns an array; Directory.EnumerateFiles is deferred (returns an IEnumerable<string>).

The Directory.EnumerateFileSystemEntries and related methods admit recursive traversal with filtering. For more elaborate filesystem operations (filtering, projection, async), the conventional patterns combine with LINQ.

The Path class

System.IO.Path provides path-string manipulation:

string combined  = Path.Combine("base", "sub", "file.txt");   // "base/sub/file.txt"
string fullPath  = Path.GetFullPath("relative/path");
string filename  = Path.GetFileName(path);
string dir       = Path.GetDirectoryName(path);
string ext       = Path.GetExtension(path);                    // ".txt"
string noExt     = Path.GetFileNameWithoutExtension(path);
string changed   = Path.ChangeExtension(path, ".log");

string root = Path.GetPathRoot(path);                            // "C:\\" on Windows; "/" on Unix
string temp = Path.GetTempFileName();                            // a unique temporary file
string tempDir = Path.GetTempPath();

The methods are platform-aware: separator characters are correct for the running OS.

Asynchronous I/O

Every conventional read or write has an async counterpart:

public async Task ProcessFile(string path) {
    using var reader = new StreamReader(path);
    string? line;
    while ((line = await reader.ReadLineAsync()) != null) {
        await ProcessLineAsync(line);
    }
}

For Stream, the principal async methods are:

• ReadAsync(buffer, cancellationToken) — buffer is Memory<byte> or byte[]-with-offset-and-count.
• WriteAsync(buffer, cancellationToken).
• CopyToAsync(destination, cancellationToken).

using var input  = File.OpenRead(in_path);
using var output = File.Create(out_path);
await input.CopyToAsync(output);    // copies the entire file

For File, the conventional async methods:

• File.ReadAllTextAsync(path), File.ReadAllBytesAsync(path), File.ReadAllLinesAsync(path).
• File.WriteAllTextAsync(path, contents), File.WriteAllBytesAsync(path, bytes).

Encoding and text encoding

The System.Text.Encoding class abstracts text encoding:

using System.Text;

byte[] utf8 = Encoding.UTF8.GetBytes("hello, 世界");
string back = Encoding.UTF8.GetString(utf8);

byte[] utf16 = Encoding.Unicode.GetBytes("hello");        // UTF-16 little-endian
byte[] ascii = Encoding.ASCII.GetBytes("hello");

The principal encodings:

Encoding	Description
Encoding.UTF8	UTF-8 (the conventional contemporary choice)
Encoding.Unicode	UTF-16 little-endian
Encoding.BigEndianUnicode	UTF-16 big-endian
Encoding.UTF32	UTF-32
Encoding.ASCII	7-bit ASCII
Encoding.Default	The system’s default encoding

For non-standard encodings, Encoding.GetEncoding("ISO-8859-1") produces an Encoding instance.

The StreamReader and StreamWriter constructors accept an Encoding; without an explicit one, they default to UTF-8 (in modern .NET).

System.IO.Pipelines

For high-throughput streaming I/O, System.IO.Pipelines provides a more efficient alternative to Stream:

using System.IO.Pipelines;

PipeReader reader = ...;
while (true) {
    ReadResult result = await reader.ReadAsync();
    ReadOnlySequence<byte> buffer = result.Buffer;

    if (TryParseFrame(ref buffer, out var frame)) {
        ProcessFrame(frame);
        reader.AdvanceTo(buffer.Start, buffer.End);
    } else {
        reader.AdvanceTo(buffer.Start, buffer.End);
    }

    if (result.IsCompleted) break;
}

The library is optimised for protocol parsers and high-volume network I/O; it admits zero-copy slicing through ReadOnlySequence<byte> and minimises allocations. The principal use is in framework code (Kestrel, the ASP.NET Core HTTP server, gRPC); application code rarely needs it.

Common patterns

Read entire file

string contents = await File.ReadAllTextAsync(path, Encoding.UTF8);

For very large files, prefer line-by-line:

await foreach (var line in File.ReadLinesAsync(path)) {     // .NET 7+
    ProcessLine(line);
}

// Or pre-.NET 7:
using var reader = new StreamReader(path);
string? line;
while ((line = await reader.ReadLineAsync()) != null) {
    ProcessLine(line);
}

Write entire file

await File.WriteAllTextAsync(path, contents, Encoding.UTF8);

Buffered processing of a binary file

using var stream = File.OpenRead(path);
byte[] buffer = new byte[4096];
int    read;
while ((read = await stream.ReadAsync(buffer)) > 0) {
    Process(buffer.AsSpan(0, read));
}

Copy with progress

async Task CopyWithProgress(Stream source, Stream destination, IProgress<long> progress) {
    byte[] buffer = new byte[81920];
    long   total  = 0;
    int    read;
    while ((read = await source.ReadAsync(buffer)) > 0) {
        await destination.WriteAsync(buffer.AsMemory(0, read));
        total += read;
        progress.Report(total);
    }
}

The IProgress<T> interface is the conventional .NET pattern for progress reporting; the implementation runs callbacks on the captured synchronisation context.

Atomic file replacement

string tempPath = path + ".tmp";
await File.WriteAllTextAsync(tempPath, contents);
File.Move(tempPath, path, overwrite: true);     // atomic on most filesystems

The pattern admits writing-then-renaming, which is atomic on most filesystems and avoids leaving the file in an intermediate state if the writer crashes.

Filesystem watcher

using var watcher = new FileSystemWatcher(directory) {
    NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.FileName,
    EnableRaisingEvents = true,
};

watcher.Changed += (sender, e) => {
    Console.WriteLine($"changed: {e.FullPath}");
};
watcher.Created += (sender, e) => { /* ... */ };
watcher.Deleted += (sender, e) => { /* ... */ };

FileSystemWatcher admits monitoring directory changes; the principal use is hot-reloading configuration, source files, or assets.

A note on the alternatives

Beyond the BCL, the .NET ecosystem provides:

• SharpZipLib, DotNetZip — alternative compression libraries.
• CsvHelper — CSV parsing.
• Serilog, NLog — logging frameworks (though most logging goes through ILogger from Microsoft.Extensions.Logging).
• Dapper — micro-ORM for SQL.
• MimeKit — MIME and email parsing.

For the principal use cases — file I/O, JSON, regex, HTTP, basic networking — the BCL is sufficient. Third-party libraries fill in the cases the BCL does not cover.