Skip to content

Memory: How the OS Hands Out and Protects Memory

When a process runs, it needs somewhere to keep its stuff -- the variables it's working with, the instructions it's executing, the data it loaded from a file. That "somewhere" is memory (specifically, RAM -- the fast, temporary storage your computer uses while it's on).

But here's the problem: lots of processes are running at the same time, and they all need memory. Somebody has to hand it out fairly, keep everyone's stuff separate, and deal with the situation when there's not enough to go around.

That somebody is the OS.

In PyOS, memory management is split into three files, each handling a different piece of the puzzle. Let's walk through them one at a time.

1. Physical Memory (memory/manager.py)

The Locker Analogy

Imagine a long hallway of school lockers. Every locker is exactly the same size, and they're numbered 0, 1, 2, 3, and so on. When a new student (a process) shows up at school, the principal (the OS) assigns them some lockers to store their stuff in.

In OS terms, each locker is called a frame, and the chunk of data that fits inside one frame is called a page. Pages and frames are the same size -- think of a page as the stuff, and a frame as the locker it goes into.

Why Fixed Sizes?

You might wonder: why make every locker the same size? Why not have some big lockers and some small ones?

Here's why. Imagine a hallway where lockers come in all different sizes. Over time, as students come and go, you end up with a bunch of tiny gaps scattered all over the hallway. You might have 10 small gaps, but none of them is big enough for a new student who needs a medium locker. This problem is called fragmentation -- you technically have free space, but it's chopped up into useless little pieces.

Fixed-size lockers avoid this entirely. A free locker is a free locker. Any page fits in any frame. Simple.

What Happens When a Process Finishes?

When a process is done running, the OS takes back all of its lockers and marks them as free. Now those frames are available for the next process that needs them. Nothing is wasted.

What If All the Lockers Are Full?

If every single frame is in use and a new process asks for memory, the OS has a choice: it can either refuse (raising an OutOfMemoryError, which crashes that process) or it can try to make room using swap space, which we'll get to in a moment.

2. Virtual Memory (memory/virtual.py)

The Personal Map Analogy

Here's where things get clever.

Every student gets their own personal map of the locker hallway. On this map, it says things like:

  • "My locker 0 is actually real locker #47"
  • "My locker 1 is actually real locker #12"
  • "My locker 2 is actually real locker #83"

The student only ever sees their own simple numbering -- 0, 1, 2, 3 -- even though their actual lockers are scattered all over the hallway. The student doesn't need to know or care where the real lockers are. They just say "open my locker 2," and the OS quietly translates that into "go to real locker #83."

This personal map is called a page table, and the simple numbering the student sees is called a virtual address. The real, physical locker number is the physical address.

Address Translation

When a process says "I want the data at my page 3, position 5," the OS does this:

  1. Look up page 3 in that process's page table.
  2. The page table says: "Page 3 maps to frame #20."
  3. Go to frame #20 in physical memory and find position 5.

That's it. The process said "page 3, position 5" and the OS translated it to "frame 20, position 5." This translation happens every single time a process touches memory, and modern CPUs have a special cache (called a TLB) that makes this translation very fast -- like having a sticky note with the answer instead of looking it up every time.

Isolation: Keeping Students Apart

Here's the really important part. Student A and Student B both think they have a "locker 0." But Student A's locker 0 points to real locker #47, while Student B's locker 0 points to real locker #91. They're completely different physical lockers.

This means Student A can never accidentally (or intentionally) read or overwrite Student B's data. Each process lives in its own little world, with its own private numbering, completely unaware of what other processes are doing. This separation is called isolation, and it's one of the most important jobs of the OS.

Page Faults

What happens if a student tries to open a locker that isn't on their map at all? Maybe they ask for "my locker 99" but their map only goes up to locker 5.

That's called a page fault. The OS catches it and says, "Hey, you don't have a locker 99." Depending on the situation, the OS might load the missing page from disk (more on that below), or it might just stop the process with an error because it tried to access memory it doesn't own.

3. Page Replacement and Swap (memory/swap.py)

The Basement Storage Room

OK, so what happens when all the lockers in the hallway are full, but a new student still needs one?

The OS has a trick: there's a storage room in the basement. In real computers, this "basement" is your hard drive or SSD -- it's much slower than RAM, but there's a lot more of it. This system is called swap space.

Here's how it works:

  1. The OS picks a locker that's currently in use.
  2. It moves everything from that locker down to the basement storage room.
  3. Now that locker is empty, so the OS gives it to the new student.
  4. Later, when the original owner needs their stuff back, the OS finds another locker to free up, brings the stuff back from the basement, and the process continues like nothing happened.

The tricky part is step 1: which locker do you pick? You want to pick one that won't be needed again soon, so you don't have to keep shuffling things back and forth between the hallway and the basement. That shuffling is slow, and too much of it (called thrashing) makes the whole computer grind to a halt.

Three Strategies for Picking a Locker

FIFO -- First In, First Out

The simplest idea: empty whichever locker has had the same stuff in it the longest. If locker #12 was filled first, it gets emptied first.

This is easy to implement -- you just keep a list in order. But it's not always smart. Sometimes the oldest stuff is actually the stuff you use the most. Imagine emptying the locker of the student who visits it every single period just because they got it a long time ago. Not great.

LRU -- Least Recently Used

A smarter idea: empty the locker that nobody has opened in the longest time. If locker #7 hasn't been touched in three hours but locker #12 was opened five minutes ago, empty locker #7.

This usually makes better choices than FIFO, because stuff that hasn't been used in a while probably won't be used again soon. The downside? You have to keep track of every single time someone opens a locker, which takes extra work and extra bookkeeping.

Clock -- Second Chance

This is the clever compromise that most real operating systems actually use.

Picture a clock hand sweeping around a circle of lockers. Each locker has a tiny flag -- a single bit that means "I was used recently."

When the OS needs to free a locker, the clock hand starts sweeping:

  1. It looks at the locker it's pointing to.
  2. If the flag is up ("I was used recently!"), the OS puts the flag down and moves the hand to the next locker. That locker gets a second chance.
  3. If the flag is down ("I haven't been used in a while"), the OS empties that locker and uses it.

The clock hand keeps going around and around, like the hand of an actual clock. Any locker that gets used has its flag set back up. So the only lockers that get emptied are the ones that haven't been used since the last time the hand came around.

This is nearly as smart as LRU -- it avoids emptying recently-used lockers -- but it's much simpler because you only need one flag per locker instead of a full history of every access.

Demand Paging

There's one more idea worth knowing about: demand paging.

When a process starts up, the OS doesn't immediately load all of its pages into memory. That would be wasteful -- maybe the process has a ton of data but only uses a small piece of it right away. Instead, the OS waits.

When the process actually tries to read a page that isn't in memory yet, a page fault happens. But this time it's not an error -- it's expected. The OS says, "Oh, you need that page? Hold on." It pauses the process, loads the page from disk into a free frame (evicting another page if necessary), updates the page table, and then lets the process continue. The process doesn't even know it was paused.

This "load it only when you need it" approach is called demand paging, and it means programs can start faster and use less memory overall, because they only occupy frames for the pages they're actually using right now.

4. Copy-on-Write (COW)

The Shared Notebook Analogy

Imagine two students, Alice and Bob, are both studying from the same textbook. Instead of photocopying the whole book for Bob (slow and wasteful), the teacher says: "Just share. You're both reading the same pages anyway."

But there's a catch -- if Bob wants to scribble notes in the margin, he can't do it on the shared copy or he'd mess up Alice's book too. So the rule is: the first person to write on a page gets a photocopy of just that one page. Now Bob has his own copy to scribble on, and Alice's page is untouched. If neither of them ever writes, no copies are ever needed. Free sharing!

This is copy-on-write (COW), and it's one of the cleverest tricks in operating systems.

Why Does This Matter?

When a process forks (creates a child), the child needs its own copy of the parent's memory. The naive approach is to copy every single page immediately -- but that's wasteful. Many forked processes immediately call exec() to load a completely different program, throwing away all those copies before they're even read.

With COW: 1. Fork is nearly free -- parent and child point to the same physical frames. No copying happens. 2. Reads are shared -- both processes read from the same frames. No extra memory used. 3. Writes trigger a copy -- only the page being written gets duplicated. The writer gets a private copy; the other process keeps the original.

How It Works Under the Hood

  1. At fork time, the kernel marks all shared pages as COW-protected and bumps a reference count on each physical frame (from 1 to 2).
  2. On read, nothing special happens. Both processes read the same data from the same frame.
  3. On write, the virtual memory system detects the COW flag and triggers a fault handler. The handler:
  4. Allocates a brand-new frame.
  5. Copies the data from the shared frame to the new one.
  6. Updates the writer's page table to point to the new frame.
  7. Decrements the old frame's reference count.
  8. Clears the COW flag on that page (it's now private).
  9. When refcount hits 0, the frame goes back to the free pool.

Reference Counting

The memory manager keeps a counter for each frame: "how many processes are using this frame right now?" When a frame is first allocated, the count is 1. When fork shares it, the count goes to 2. If three processes share it (A forks B, then B forks C), the count is 3. Each termination or COW copy decrements the count. The frame is only truly freed when the count reaches 0.

5. Memory-Mapped Files (mmap)

The Open Textbook Analogy

Imagine you're doing homework and need to look something up in a textbook. The normal way is to grab the book, photocopy the pages you need, put the copies on your desk, read and scribble on them, and later copy your changes back into the book. That's a lot of steps!

Memory-mapped files are like opening the textbook and laying it flat on your desk. Instead of photocopying pages, you read and write directly on the open book. No copying back and forth -- the book is right there.

In OS terms, mmap takes a file and makes it appear as if it's part of a process's memory. The process can read and write those memory addresses, and the OS handles moving data between the file and RAM automatically.

Two Modes: Private vs Shared

MAP_PRIVATE -- Your own photocopy

With MAP_PRIVATE, the OS hands you your own copy of the pages. You can scribble all over them, but your changes stay with you -- the original book is untouched. This is useful when a process needs to read file data into memory but doesn't want to modify the actual file.

When a process forks, private mmap pages get the same copy-on-write treatment as regular memory pages -- parent and child share until one of them writes.

MAP_SHARED -- Everyone reads the same book

With MAP_SHARED, multiple processes all look at the same physical pages. If one process writes "HELLO" at the top of page 3, every other process mapping that same file immediately sees "HELLO" there too -- because they're all looking at the same underlying memory.

This is powerful for communication between processes. Instead of sending messages back and forth, they can just read and write the same shared memory.

When a process forks, shared mmap pages are not marked copy-on-write. Writes by the parent or child should be visible to the other -- that's the whole point of "shared."

msync -- Saving Your Work

Shared mappings live in memory, but the actual file on disk doesn't update automatically. When you want to make sure your changes are saved to the file, you call msync -- think of it as pressing "Save." The OS takes whatever is in the shared memory pages and writes it back to the file.

When a process unmaps a shared region (or terminates), the OS does an automatic msync to make sure nothing is lost.

How It Works Under the Hood

  1. mmap -- The kernel reads the file data, allocates physical frames, copies the data into those frames, and maps them into the process's virtual address space. For shared mappings, a second process mapping the same file reuses the same frames (zero extra allocation).

  2. Read/write -- The process just reads and writes memory addresses like normal. For private mappings, changes stay local. For shared mappings, changes are visible to all sharers.

  3. munmap -- Removes the mapping. For shared regions, data is written back to the file first. Frames are released and reference counts updated.

  4. msync -- Writes shared data back to the file without removing the mapping. Like saving a document without closing it.

6. Slab Allocator (memory/slab.py)

The Supply Closet Analogy

Imagine the school warehouse full of big storage crates (frames). Every time a teacher needs a single pencil, they get an entire crate. That's a huge waste of space!

A slab allocator is like setting up a supply closet. You take one crate from the warehouse and divide it into little compartments -- one pencil per compartment. Now when a teacher needs a pencil, they grab one from the supply closet in a split second. When they're done, they put it back. No waste, no waiting.

Each supply closet handles one type of item. The "pencil closet" has pencil-sized compartments, the "eraser closet" has eraser-sized ones. In OS terms, each closet is a slab cache, each compartment is a slot, and the crate backing it is a slab (one physical frame).

Why Not Just Use Frames?

The memory manager allocates in whole frames (256 bytes each). But kernel objects like process control blocks (PCBs, ~64 bytes) and inodes (~48 bytes) are much smaller. If you use a whole frame for each one, you waste 75-80% of the space. Multiply that by hundreds of objects and the waste adds up fast.

A slab allocator solves this by packing many small objects into a single frame. A 256-byte frame with 64-byte slots holds 4 PCBs. Same space, four times the capacity.

The Three-Level Hierarchy

The slab allocator has three levels, from bottom to top:

  1. Slab -- one physical frame divided into equal-sized slots. A slab for 64-byte objects has 4 slots (256 / 64). It tracks which slots are free using a simple stack: pop to allocate, push to free. Both operations are O(1) -- constant time, no matter how many slots there are.

  2. Slab Cache -- a pool of slabs, all for the same object size. The "pcb" cache might have three slabs (three frames), giving 12 total slots for PCBs. When all slabs are full, the cache auto-grows by requesting another frame from the memory manager.

  3. Slab Allocator -- a registry of named caches. The kernel creates caches for each type of object it needs: "pcb" for process control blocks, "inode" for filesystem metadata, and so on. User code talks to the allocator by cache name.

How Allocation Works

When the kernel needs a new PCB:

  1. It asks the slab allocator for an object from the "pcb" cache.
  2. The allocator finds the "pcb" slab cache.
  3. The cache scans its slabs for one with free space.
  4. If a slab has a free slot, it pops the slot index from its free stack. Done!
  5. If all slabs are full, the cache asks the memory manager for a new frame, creates a new slab, and allocates from that.

The result is a handle: (slab_index, slot_index) -- exactly which frame and which slot your object lives in. It's transparent and inspectable, which makes it great for learning.

Auto-Growing

When all slabs in a cache are full and a new allocation comes in, the cache automatically requests another frame from the memory manager. This is like the supply closet running out of pencils -- you grab another crate from the warehouse and add more compartments.

In real operating systems, slab allocators can also shrink by returning empty slabs back to the page allocator. Our simulator keeps things simple and doesn't reclaim empty slabs.

How It Works Under the Hood

  1. At boot, the kernel creates a SlabAllocator backed by the memory manager. It pre-registers caches for common objects ("pcb" at 64 bytes, "inode" at 48 bytes).

  2. On allocate, the cache pops a free slot index from one of its slabs. If no slab has space, it requests a new frame and creates a new slab.

  3. On free, the slot index is pushed back onto the slab's free stack. The slot is immediately available for the next allocation.

  4. On destroy, all backing frames are returned to the memory manager.

7. Demand Paging in PyOS: The Library Desk

The Analogy

Imagine you're studying in the library. There's a huge shelf of books (virtual memory), but your desk (physical RAM) only fits a handful of books at a time.

When you need a book that's already on your desk, great -- just grab it. But when you need a book that's on the shelf, you have to make room. You pick a book on your desk that you're not reading right now, put it back on the shelf, and grab the new one. That swap -- putting one back, pulling one out -- is exactly what the OS does.

In PyOS, the Pager is the librarian managing your desk. It:

  1. Keeps track of which "books" (pages) are on your desk (resident in RAM).
  2. Notices when you ask for a book that isn't there (a page fault).
  3. Picks a book to shelve using one of three strategies (FIFO, LRU, or Clock).
  4. Swaps them -- saves the old page to swap space, loads the new one into RAM.

Try It Yourself

swap              # See current swap status
swap policy fifo  # Switch to FIFO replacement
swap demo         # Force page faults and see what happens
swap policy lru   # Switch back to LRU
swap demo         # Compare the results

The swap demo command writes to more virtual pages than fit in physical RAM, forcing the pager to evict and reload pages. Watch how the number of page faults changes with different policies!

Putting It All Together

Here's how these pieces work as a team:

  1. Physical memory (memory/manager.py) manages the actual frames -- the real lockers in the hallway. It tracks which ones are free and which ones are taken.

  2. Virtual memory (memory/virtual.py) gives each process its own private map (page table) so that processes see clean, simple addresses and can't interfere with each other.

  3. Swap (memory/swap.py) handles the overflow. When memory is full, it moves pages to disk and brings them back when needed, using a replacement strategy (FIFO, LRU, or Clock) to decide what to move.

  4. Slab allocator (memory/slab.py) sits on top of the frame allocator and subdivides frames into small, fixed-size slots for kernel objects. It eliminates waste when many tiny objects need their own space.

Together, they create the illusion that every process has its own big, private chunk of memory -- even if the physical RAM is small and shared among many processes. That illusion is one of the most powerful tricks in all of computing.

Where to Go Next

  • Processes -- How the OS runs programs and shares the processor
  • What Is an OS? -- The big picture of how an OS works
  • Filesystem -- How files and folders are organized and stored