OSTEP 24
This commit is contained in:
ridethepig
parent
4847f678b3
commit
c64c66c73f
File diff suppressed because it is too large
Load Diff
@ -1014,16 +1014,330 @@ file-path:: ../assets/ostep_1680491762166_0.pdf
|
|||||||
ls-type:: annotation
|
ls-type:: annotation
|
||||||
id:: 64319715-78ae-4efe-bab1-def007ee8e78
|
id:: 64319715-78ae-4efe-bab1-def007ee8e78
|
||||||
hl-color:: yellow
|
hl-color:: yellow
|
||||||
- Reserve some space on disk for moving pages back and forth, and the OS needs to remember the disk address of a given page.
|
- Reserve some space on disk for moving pages back and forth, and the OS needs to remember the *disk address* of a given page.
|
||||||
- swap space is not the only on-disk location for swapping, e.g. program binary loading, not necessarily load the whole code segment at first
|
- swap space is not the only on-disk location for swapping, e.g. program binary loading, not necessarily load the whole code segment at first
|
||||||
hl-page:: 258
|
hl-page:: 258
|
||||||
ls-type:: annotation
|
ls-type:: annotation
|
||||||
id:: 64319809-1dbe-48dc-a4e4-7e14062d42c5
|
id:: 64319809-1dbe-48dc-a4e4-7e14062d42c5
|
||||||
hl-color:: yellow
|
hl-color:: yellow
|
||||||
-
|
- Present bit and Page fault
|
||||||
-
|
hl-page:: 259
|
||||||
- chinery
|
ls-type:: annotation
|
||||||
|
id:: 643221b9-c6ee-4cdd-bde9-6d31cd39f9f3
|
||||||
|
hl-color:: yellow
|
||||||
|
- PTE has a present bit which indicates whether the page is in memory. When accessing a page with present bit clear (of course, assume valid), hardware raises a page fault exception.
|
||||||
|
- One way to keep the *disk address* is to reuse the original PFN field(after all, when a page is on disk, PFN is invalid)
|
||||||
|
- while the I/O is in flight, the process will be in the blocked state
|
||||||
|
hl-page:: 261
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 6432277e-131b-4651-a2e9-bd51e4e5b7e6
|
||||||
|
hl-color:: yellow
|
||||||
|
- Page-Fault Control Flow Algorithm (Software)
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 263
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 643229bd-fc80-45a7-a326-ddb73c8df04f
|
||||||
|
- What if the memory if full?
|
||||||
|
- *page out* one or more pages to make room for the new page(s)
|
||||||
|
- In practice, OS always wants to keep a small amount of memory free. Most OSs have some kind of high watermark (HW ) and low watermark (LW ) to help decide when to start evicting pages from memory.
|
||||||
|
hl-page:: 263
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 64322a2c-91e1-47b7-8c3a-174d1c718b9f
|
||||||
|
hl-color:: yellow
|
||||||
|
- When the OS notices that there are fewer than LW pages available, a background thread that is responsible for freeing memory runs.
|
||||||
|
hl-page:: 263
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 64322b0d-6e56-44aa-a5ec-a23e43355053
|
||||||
|
hl-color:: yellow
|
||||||
|
The thread evicts pages until there are HW pages available.
|
||||||
|
The background thread then goes to sleep.
|
||||||
|
- Possible optimization: cluster or group a number of pages and write them out at once for better disk efficiency
|
||||||
|
- proactive 积极主动的；主动出击的；先发制人的
|
||||||
|
hl-page:: 263
|
||||||
ls-type:: annotation
|
ls-type:: annotation
|
||||||
hl-page:: 259
|
id:: 6432298b-31e8-419e-b84d-3ecf08616a25
|
||||||
hl-color:: green
|
hl-color:: green
|
||||||
id:: 64319857-9885-49e0-9e51-c974f0b6b038
|
- Replacement Policy
|
||||||
|
hl-page:: 268
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 64324655-73cb-40a7-814b-1dd26b310e41
|
||||||
|
hl-color:: yellow
|
||||||
|
- cache: RAM can be regarded as the *cache* for all the pages in the system. And the goal of our replacement policy is to minimize cache misses.
|
||||||
|
hl-page:: 268
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 6432466b-0d2b-4dac-990f-168a82743a94
|
||||||
|
hl-color:: yellow
|
||||||
|
- Average Memory Access Time (AMAT): $AMAT = T_M + P_{Miss} \cdot T_D$, where $T_M$ is the cost of memory access and $T_D$ is the cost of disk access. The cost of disk access is so high that *miss rate* is crucial to the overall AMAT.
|
||||||
|
hl-page:: 268
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 6432471f-f8f1-4a2c-aa82-abd0ec24142e
|
||||||
|
hl-color:: yellow
|
||||||
|
- Optimal Replacement Policy
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 269
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 643248c0-69b7-4b23-a8be-94df2abb38e6
|
||||||
|
- Replace the page that will *be accessed furthest in the future*.
|
||||||
|
- The reason this is true is simple: you will refer to the other pages before you refer to the one furthest out.
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 270
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 64324c4e-165b-4ba4-838a-afc002decd34
|
||||||
|
- FIFO: evict the earliest page, cannot determine the importance of blocks
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 271
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 64325410-09a2-472f-b67b-426df53d4c81
|
||||||
|
- Random: randomly choose one page to evict, depend on the luck
|
||||||
|
hl-page:: 273
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 64325422-d727-4104-9c85-6c18260a1b6e
|
||||||
|
hl-color:: yellow
|
||||||
|
- LRU
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 274
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 643256d9-6c65-4e29-adab-f5a6d112bccd
|
||||||
|
- Historical information: frequency and recency of access. Thus there is Least-Frequently-Used (LFU) policy and Least-Recently-Used (LRU) policy
|
||||||
|
- LRU Implement
|
||||||
|
hl-page:: 278
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 64325b27-c557-40ad-86ff-e46eb11dbf77
|
||||||
|
hl-color:: yellow
|
||||||
|
- Precise LRU: On every memory reference, some data structure need to be updated to maintain the LRU property, which is slow even with some hardware support.
|
||||||
|
- Approximating LRU:
|
||||||
|
- Hardware support: *use bit* in PTE. Whenever a page is referenced, the *use bit* is set to 1 and the hardware never clears it.
|
||||||
|
- clock algorithm: All pages are stored in a circular structure, and a *clock hand* points to one of the pages. On eviction demand, check the *used bit* of the current page. If set, clear it and go to the next page, until a page with *used bit* is 0 (it guarantees to converge, the worst case is to search through all pages and clear them all).
|
||||||
|
hl-page:: 279
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 6432650c-142d-46ba-a8b1-4f0c1af50635
|
||||||
|
hl-color:: yellow
|
||||||
|
- Indeed, any approach which periodically clears the *use bits* and then differentiates between which pages have *use bits* of 1 versus 0 to decide which to replace would be fine
|
||||||
|
hl-page:: 279
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 64326b23-8bc8-4b41-96ac-d4044db00dbd
|
||||||
|
hl-color:: yellow
|
||||||
|
- Workload Examples
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 275
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 643258e8-576b-4e22-91da-8579517b82bc
|
||||||
|
- **No-locality workload**: each reference is to a random page within the set of accessed pages. Optimal do better, and the others are almost the same.
|
||||||
|
- **80-20 workload**: 80% of the references are made to 20% of the pages.
|
||||||
|
- **Looping-sequential workload**: worst case for FIFO and LRU, because they keep evicting older pages which will be accessed sooner than cached pages.
|
||||||
|
Random does better, one of its nice properties is not having weird corner-case behaviors.
|
||||||
|
*scan resistance* is added to LRU to try to avoid this worst case for LRU.
|
||||||
|
- **Dirty Bit**: Dirty pages may be more costly to evict, because they need to be written to disk instead of simply discarded. Therefore, some systems prefer to evict clean pages over dirty pages.
|
||||||
|
- **thrashing**
|
||||||
|
hl-page:: 281
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 64339732-f900-4f7b-8783-782c8cb73550
|
||||||
|
hl-color:: yellow
|
||||||
|
- The system will constantly be paging, when the running process requires more memory than the available physical memory.
|
||||||
|
- **admission control**: not to run some subset of the processes when they require too much memory
|
||||||
|
hl-page:: 281
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 643398d2-0574-49f6-9873-c3b725d3649f
|
||||||
|
hl-color:: yellow
|
||||||
|
- TYPES OF CACHE MISSES:
|
||||||
|
hl-page:: 271
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 64324c6c-2057-46b3-aa2f-fa1e0748e1e8
|
||||||
|
hl-color:: yellow
|
||||||
|
- compulsory miss: cold-start miss
|
||||||
|
- capacity miss
|
||||||
|
- conflict miss: set-associativity, limits on where an item can be placed
|
||||||
|
- anomaly 异常事物；反常现象
|
||||||
|
hl-page:: 272
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 6432503c-9c30-4ae0-8aa0-e867554b1662
|
||||||
|
hl-color:: green
|
||||||
|
- discrepancy 差异；不符合；不一致
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 282
|
||||||
|
hl-color:: green
|
||||||
|
id:: 64339651-5357-49a3-b495-6f1db7826dcc
|
||||||
|
- prohibitive 费用或价格）高得令人望而却步的；限制性的，禁止的；
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 282
|
||||||
|
hl-color:: green
|
||||||
|
id:: 64339660-fb5e-46f9-94a1-bd70bc684fc5
|
||||||
|
- thrash 翻来覆去；抽打；一举战胜 （不是 trash 垃圾）
|
||||||
|
hl-page:: 281
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 64339700-b057-4150-bc27-1a3ee4cb536d
|
||||||
|
hl-color:: green
|
||||||
|
- VAX/VMS Virtual Memory
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 286
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 64339ec6-877f-42eb-ba92-4211f543c0f3
|
||||||
|
- Memory Management Hardware
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 286
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 64339efd-a5cd-487b-9eb8-caf7e824de7a
|
||||||
|
- 32 bit VA and 512B page -> 2 bit segment, 21 bit VPN, 9 bit offset
|
||||||
|
- 4 segments: P0, P1, S0, S1
|
||||||
|
- Process Space: lower half of the AS, unique for each process. Divided into `P0` and `P1`, `P0` includes Code and Heap, while `P1` is Stack.
|
||||||
|
- System Space: upper half of AS, though only half of this is used by the VMS OS.
|
||||||
|
- Problem: 512-Byte page is so small that page table could occupy too much memory.
|
||||||
|
- Per-segment page table, VAX-11 provides each segment with a pair of base-and-bounds register
|
||||||
|
- The page tables are allocated in OS memory(System Space), though this complicates the translation process.
|
||||||
|
- The VAX/VMS Address Space
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 288
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 6433a3cc-c528-4c8d-bca2-9bdab780e27a
|
||||||
|
- Invalid Page 0: help debug null-pointers
|
||||||
|
- base and bounds registers for S0 (OS) stay the same and the kernel is mapped into each process's AS
|
||||||
|
- Page Replacement
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 289
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 6433a4e2-191f-4c49-8c5f-e9a78cb4c476
|
||||||
|
- No reference bit: VAX-11 doesn't have this. Fortunately, just what we does in COW, this can be emulated by page fault.
|
||||||
|
- Memory hogging: LRU is not a fair share policy.
|
||||||
|
- Segmented FIFO:
|
||||||
|
- Resident Set Size(RSS): each process has a maximum number of pages it can keep in memory
|
||||||
|
- FIFO list: when a process exceeds its RSS, the first page is evicted
|
||||||
|
- Second-chance lists: a *global* clean-page list and a *global* dirty-page list. When evicted, page is removed from per-process FIFO and added to these 2 lists. If another process needs a free page, give a clean page from this list. Or, if the original process raises a page fault, it reclaims a page.
|
||||||
|
- **clustering**: VMS groups large batches of pages together from the global dirty list, and writes them to disk in one fell swoop.
|
||||||
|
hl-page:: 290
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 6433a931-90cf-4e1f-8ea7-922bc81f354e
|
||||||
|
hl-color:: yellow
|
||||||
|
- Lazy optimizations: use protection and trap to save some laborious work
|
||||||
|
hl-page:: 291
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 6433a9cd-4ecd-4a8e-b778-8c0c36af929a
|
||||||
|
hl-color:: yellow
|
||||||
|
- **demand zeroing**: The OS is responsible to zero a page when allocating it to a process for security and isolation. This can be deferred to when the process actually uses this page in case the process never uses it.
|
||||||
|
hl-page:: 291
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 6433aa48-cb30-4a01-9670-64ac357bc2ab
|
||||||
|
hl-color:: yellow
|
||||||
|
- **copy-on-write**: When the OS needs to copy a page from one address space to another, instead of copying it, it can map it into the target address space and mark it read-only in both address spaces.
|
||||||
|
hl-page:: 291
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 6433aa4c-e737-43aa-bcc9-34609d86fe42
|
||||||
|
hl-color:: yellow
|
||||||
|
- circumvent 规避；设法回避；绕过；绕行
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 287
|
||||||
|
hl-color:: green
|
||||||
|
id:: 6433a0cd-3e8b-4c0c-af4e-8cbf498b9231
|
||||||
|
- astute 精明的；狡猾的
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 289
|
||||||
|
hl-color:: green
|
||||||
|
id:: 6433a2a0-7ba4-46cf-b1ec-fc02ce4c7916
|
||||||
|
- hog
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 289
|
||||||
|
hl-color:: green
|
||||||
|
id:: 6433a4fb-da6c-4741-a962-6c95c7fd9f0c
|
||||||
|
- swoop 俯冲；突然袭击；突击搜查；突然行动
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 291
|
||||||
|
hl-color:: green
|
||||||
|
id:: 6433a96f-0414-4ec5-963b-72161ce23ef0
|
||||||
|
- The Linux Virtual Memory System
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 292
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 6433abf7-a59b-42ae-af0b-05706abee681
|
||||||
|
- The Linux Address Space
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 293
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 6433ac19-794e-4bcb-abf9-d406d1a148d3
|
||||||
|
- **kernel logical addresses**
|
||||||
|
- allocated by `kmalloc`
|
||||||
|
- most kernel data structures live here like page tables and per-process kernel stacks
|
||||||
|
- cannot be swapped to disk
|
||||||
|
- linear mapped to physical memory, `PA = VA - 0xC000_0000`, which simplifies translation and its *contiguity* for DMA
|
||||||
|
- kernel virtual address
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 294
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 6433b05c-6a5a-4ae5-8664-03fa83483598
|
||||||
|
- allocated by `vmalloc`
|
||||||
|
- no physical contiguity guarantee
|
||||||
|
- easier to map and able to allocate larger size
|
||||||
|
- Page Table
|
||||||
|
- 64-bit page table structure: 48 bit VA, including 36 bit 4 level VPN and 12 bit offset for 4KB page size
|
||||||
|
- Large Page Support
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 295
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 6433b26b-111b-4fa3-930b-232d23287999
|
||||||
|
- Intel x86 allows for use of multiple page sizes
|
||||||
|
- Advantages: less memory consumption from page table, fewer TLB miss, shorter TLB miss path
|
||||||
|
- Disadvantages: internal fragmentation, more cost in swapping
|
||||||
|
- Explicit interface: applications explicitly request memory allocated with large pages through calls like `mmap` or `shmget`
|
||||||
|
- Transparent huge page support: When this feature is enabled, the operating system automatically looks for opportunities to allocate huge pages without requiring application modification.
|
||||||
|
- The Page Cache
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 297
|
||||||
|
hl-color:: yellow
|
||||||
|
id:: 6433b63d-6815-4c33-a5fb-c7c20c06fcab
|
||||||
|
- To reduce costs of accessing persistent storage
|
||||||
|
- unified, keeping pages in memory from 3 primary sources: 1. memory-mapped files; 2. file data and metadata from devices; 3. heap and stack pages that comprise each process.
|
||||||
|
hl-page:: 297
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 6433b736-af15-4e56-b2ba-3bec294483ce
|
||||||
|
hl-color:: yellow
|
||||||
|
- Dirty data is periodically written to the backing store by background threads
|
||||||
|
- 2Q replacement algorithm
|
||||||
|
hl-page:: 297
|
||||||
|
ls-type:: annotation
|
||||||
|
id:: 6433b6dc-b1d4-4c88-bd92-8c9359bece02
|
||||||
|
hl-color:: yellow
|
||||||
|
- Solved problem: cyclic access to a large file will kick out other pages
|
||||||
|
- Divide the pages into 2 lists: *inactive* list and *active* list
|
||||||
|
- accessed for the first time, a page is placed on the *inactive* list
|
||||||
|
re-referenced, the page is promoted to the *active* list
|
||||||
|
periodically move bottom of the *active* list to *inactive* list
|
||||||
|
within the 2 lists, manage pages in LRU order
|
||||||
|
- When replacing, the candidate is taken from the *inactive* list.
|
||||||
|
- tract
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 295
|
||||||
|
hl-color:: green
|
||||||
|
id:: 6433b29c-817e-44f0-a02e-8ab14f6fe764
|
||||||
|
- yell
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 296
|
||||||
|
hl-color:: green
|
||||||
|
id:: 6433b321-04d6-4118-9d04-80758598654e
|
||||||
|
- scorn
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 296
|
||||||
|
hl-color:: green
|
||||||
|
id:: 6433b345-d17f-48ea-b0d4-ec3eba091be4
|
||||||
|
- subverted
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 297
|
||||||
|
hl-color:: green
|
||||||
|
id:: 6433b6c0-0673-41c5-93a5-8280f5a6ab4b
|
||||||
|
- speculative
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 301
|
||||||
|
hl-color:: green
|
||||||
|
id:: 6433b625-8749-4211-8d7a-3770dcd86ec2
|
||||||
|
- multitude
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 302
|
||||||
|
hl-color:: green
|
||||||
|
id:: 6433b5fa-2c4f-434c-9ab8-5142eb2cec45
|
||||||
|
- shudder
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 305
|
||||||
|
hl-color:: green
|
||||||
|
id:: 6433bb38-56f9-47b1-b4d1-617c8693eb50
|
||||||
|
- cheapskate
|
||||||
|
ls-type:: annotation
|
||||||
|
hl-page:: 306
|
||||||
|
hl-color:: green
|
||||||
|
id:: 6433bce5-e05b-4fd3-8af1-30bdd091ab00
|
||||||
Loading…
Reference in New Issue
Block a user