#include <MemStore.h>

Inheritance diagram for MemStore:

Collaboration diagram for MemStore:

Classes
class	SlotAndPage
	temporary storage for slot and page ID pointers; for the waiting cache More...

Public Member Functions
	MemStore ()

	~MemStore () override

bool	keepInLocalMemory (const StoreEntry &e) const
	whether e should be kept in local RAM for possible future caching

void	write (StoreEntry &e)
	copy non-shared entry data of the being-cached entry to our cache

void	completeWriting (StoreEntry &e)
	all data has been received; there will be no more write() calls

void	disconnect (StoreEntry &e)
	called when the entry is about to forget its association with mem cache

void	create () override
	create system resources needed for this store to operate in the future

void	init () override

StoreEntry *	get (const cache_key *) override

uint64_t	maxSize () const override

uint64_t	minSize () const override
	the minimum size the store will shrink to via normal housekeeping

uint64_t	currentSize () const override
	current size

uint64_t	currentCount () const override
	the total number of objects stored right now

int64_t	maxObjectSize () const override
	the maximum size of a storable object; -1 if unlimited

void	getStats (StoreInfoStats &stats) const override
	collect statistics

void	stat (StoreEntry &e) const override

void	reference (StoreEntry &e) override
	somebody needs this entry (many cache replacement policies need to know)

bool	dereference (StoreEntry &e) override

void	updateHeaders (StoreEntry *e) override
	make stored metadata and HTTP headers the same as in the given entry

void	maintain () override
	perform regular periodic maintenance; TODO: move to UFSSwapDir::Maintain

bool	anchorToCache (StoreEntry &) override

bool	updateAnchored (StoreEntry &) override

void	evictCached (StoreEntry &) override

void	evictIfFound (const cache_key *) override

virtual int	callback ()
	called once every main loop iteration; TODO: Move to UFS code.

virtual void	sync ()
	prepare for shutdown

Static Public Member Functions
static bool	Enabled ()
	whether Squid is correctly configured to use a shared memory cache

static int64_t	EntryLimit ()
	calculates maximum number of entries we need to store and map

static bool	Requested ()

Protected Member Functions
bool	shouldCache (StoreEntry &e) const
	whether we should cache the entry

bool	startCaching (StoreEntry &e)
	locks map anchor and preps to store the entry in shared memory

void	copyToShm (StoreEntry &e)
	copies all local data to shared memory

void	copyToShmSlice (StoreEntry &e, Ipc::StoreMapAnchor &anchor, Ipc::StoreMap::Slice &slice)
	copies at most one slice worth of local memory to shared memory

bool	copyFromShm (StoreEntry &e, const sfileno index, const Ipc::StoreMapAnchor &anchor)
	copies the entire entry from shared to local memory

void	copyFromShmSlice (StoreEntry &, const StoreIOBuffer &)
	imports one shared memory slice into local memory

void	updateHeadersOrThrow (Ipc::StoreMapUpdate &update)

void	anchorEntry (StoreEntry &e, const sfileno index, const Ipc::StoreMapAnchor &anchor)
	anchors StoreEntry to an already locked map entry

bool	updateAnchoredWith (StoreEntry &, const sfileno, const Ipc::StoreMapAnchor &)
	updates Transients entry after its anchor has been located

Ipc::Mem::PageId	pageForSlice (Ipc::StoreMapSliceId sliceId)
	safely returns a previously allocated memory page for the given entry slice

Ipc::StoreMap::Slice &	nextAppendableSlice (const sfileno entryIndex, sfileno &sliceOffset)

sfileno	reserveSapForWriting (Ipc::Mem::PageId &page)
	finds a slot and a free page to fill or throws

void	noteFreeMapSlice (const Ipc::StoreMapSliceId sliceId) override
	adjust slice-linked state before a locked Readable slice is erased

Protected Attributes
friend	ShmWriter

Private Types
typedef MemStoreMapExtras	Extras

Private Attributes
Ipc::Mem::Pointer< Ipc::Mem::PageStack >	freeSlots
	unused map slot IDs

MemStoreMap *	map
	index of mem-cached entries

Ipc::Mem::Pointer< Extras >	extras
	IDs of pages with slice data.

sfileno	lastWritingSlice
	the last allocate slice for writing a store entry (during copyToShm)

SlotAndPage	waitingFor
	a cache for a single "hot" free slot and page

Detailed Description

Stores HTTP entities in RAM. Current implementation uses shared memory. Unlike a disk store (SwapDir), operations are synchronous (and fast).

Definition at line 29 of file MemStore.h.

Member Typedef Documentation

◆ Extras

typedef MemStoreMapExtras MemStore::Extras

private

Definition at line 102 of file MemStore.h.

Constructor & Destructor Documentation

◆ MemStore()

MemStore::MemStore ( )

Definition at line 164 of file MemStore.cc.

◆ ~MemStore()

MemStore::~MemStore ( )

override

Definition at line 168 of file MemStore.cc.

References map.

Member Function Documentation

◆ anchorEntry()

void MemStore::anchorEntry	(	StoreEntry &	e,
		const sfileno	index,
		const Ipc::StoreMapAnchor &	anchor
	)

protected

Definition at line 454 of file MemStore.cc.

References assert, Ipc::StoreMapAnchor::complete(), EBIT_SET, ENTRY_VALIDATED, Ipc::StoreMapAnchor::exportInto(), StoreEntry::flags, StoreEntry::hasDisk(), IN_MEMORY, MemObject::MemCache::index, MemObject::MemCache::io, Store::ioReading, StoreEntry::mem_obj, MemObject::memCache, NOT_IN_MEMORY, MemObject::object_sz, StoreEntry::setMemStatus(), STORE_OK, STORE_PENDING, StoreEntry::store_status, and StoreEntry::swap_file_sz.

Referenced by anchorToCache(), and get().

◆ anchorToCache()

bool MemStore::anchorToCache ( StoreEntry & )

overridevirtual

tie StoreEntry to this storage if this storage has a matching entry

Return values

true	if this storage has a matching entry

Reimplemented from Store::Controlled.

Definition at line 410 of file MemStore.cc.

References anchorEntry(), Assure, StoreEntry::hasMemStore(), Here, MemObject::MemCache::io, Store::ioDone, hash_link::key, map, StoreEntry::mem(), MemObject::memCache, Ipc::StoreMap::openForReading(), and updateAnchoredWith().

◆ callback()

virtual int Store::Storage::callback ( )

inlinevirtualinherited

Reimplemented in Fs::Ufs::UFSSwapDir, Store::Controller, and Store::Disks.

Definition at line 72 of file Storage.h.

◆ completeWriting()

void MemStore::completeWriting ( StoreEntry & e )

Definition at line 894 of file MemStore.cc.

References assert, CollapsedForwarding::Broadcast(), Ipc::StoreMap::closeForWriting(), debugs, MemObject::MemCache::index, MemObject::MemCache::io, Store::ioDone, map, StoreEntry::mem_obj, MemObject::memCache, MemObject::MemCache::offset, and StoreEntry::storeWriterDone().

Referenced by write().

◆ copyFromShm()

bool MemStore::copyFromShm	(	StoreEntry &	e,
		const sfileno	index,
		const Ipc::StoreMapAnchor &	anchor
	)

protected

◆ copyFromShmSlice()

void MemStore::copyFromShmSlice	(	StoreEntry &	e,
		const StoreIOBuffer &	buf
	)

protected

Definition at line 575 of file MemStore.cc.

References assert, MemObject::data_hdr, debugs, MemObject::endOffset(), StoreIOBuffer::length, StoreEntry::mem_obj, StoreIOBuffer::offset, and mem_hdr::write().

Referenced by copyFromShm().

◆ copyToShm()

void MemStore::copyToShm ( StoreEntry & e )

protected

Definition at line 680 of file MemStore.cc.

References assert, copyToShmSlice(), debugs, EBIT_TEST, MemObject::endOffset(), ENTRY_FWD_HDR_WAIT, StoreEntry::flags, MemObject::MemCache::index, lastWritingSlice, map, maxObjectSize(), StoreEntry::mem_obj, MemObject::memCache, Must, nextAppendableSlice(), MemObject::MemCache::offset, Ipc::StoreMapAnchor::start, and Ipc::StoreMap::writeableEntry().

Referenced by write().

◆ copyToShmSlice()

void MemStore::copyToShmSlice	(	StoreEntry &	e,
		Ipc::StoreMapAnchor &	anchor,
		Ipc::StoreMap::Slice &	slice
	)

protected

Definition at line 716 of file MemStore.cc.

References Ipc::StoreMapAnchor::basics, mem_hdr::copy(), MemObject::data_hdr, debugs, lastWritingSlice, StoreEntry::mem_obj, MemObject::memCache, MemObject::MemCache::offset, pageForSlice(), Ipc::Mem::PageSize(), Ipc::StoreMapSlice::size, Ipc::StoreMapAnchor::Basics::swap_file_sz, and TexcHere.

Referenced by copyToShm().

◆ create()

void MemStore::create ( )

inlineoverridevirtual

Implements Store::Storage.

Definition at line 48 of file MemStore.h.

◆ currentCount()

uint64_t MemStore::currentCount ( ) const

overridevirtual

Implements Store::Storage.

Definition at line 288 of file MemStore.cc.

References Ipc::StoreMap::entryCount(), and map.

Referenced by getStats(), and stat().

◆ currentSize()

uint64_t MemStore::currentSize ( ) const

overridevirtual

Implements Store::Storage.

Definition at line 281 of file MemStore.cc.

References Ipc::Mem::PageId::cachePage, Ipc::Mem::PageLevel(), and Ipc::Mem::PageSize().

Referenced by stat().

◆ dereference()

bool MemStore::dereference ( StoreEntry & e )

overridevirtual

somebody no longer needs this entry (usually after calling reference()) return false iff the idle entry should be destroyed

Implements Store::Controlled.

Definition at line 305 of file MemStore.cc.

◆ disconnect()

void MemStore::disconnect ( StoreEntry & e )

Definition at line 938 of file MemStore.cc.

References Ipc::StoreMap::abortWriting(), assert, CollapsedForwarding::Broadcast(), Ipc::StoreMap::closeForReading(), StoreEntry::hasMemStore(), MemObject::MemCache::index, MemObject::MemCache::io, Store::ioDone, Store::ioReading, Store::ioWriting, map, StoreEntry::mem_obj, MemObject::memCache, and StoreEntry::storeWriterDone().

Referenced by copyFromShm(), evictCached(), and write().

◆ Enabled()

static bool MemStore::Enabled ( )

inlinestatic

Definition at line 68 of file MemStore.h.

References EntryLimit().

Referenced by MemStoreRr::create(), Store::Controller::init(), StoreEntry::setMemStatus(), and Store::Controller::SmpAware().

◆ EntryLimit()

int64_t MemStore::EntryLimit ( )

static

Definition at line 966 of file MemStore.cc.

References Config, SquidConfig::memMaxSize, Ipc::Mem::PageSize(), and Requested().

Referenced by MemStoreRr::claimMemoryNeeds(), MemStoreRr::create(), Enabled(), and init().

◆ evictCached()

void MemStore::evictCached ( StoreEntry & e )

overridevirtual

Prevent new get() calls from returning the matching entry. If the matching entry is unused, it may be removed from the store now. The store entry is matched using either e attachment info or e.key.

Implements Store::Storage.

Definition at line 912 of file MemStore.cc.

References CollapsedForwarding::Broadcast(), debugs, StoreEntry::destroyMemObject(), disconnect(), evictIfFound(), Ipc::StoreMap::freeEntry(), StoreEntry::hasMemStore(), MemObject::MemCache::index, StoreEntry::locked(), map, StoreEntry::mem_obj, MemObject::memCache, and StoreEntry::publicKey().

◆ evictIfFound()

void MemStore::evictIfFound ( const cache_key * )

overridevirtual

An evictCached() equivalent for callers that did not get() a StoreEntry. Callers with StoreEntry objects must use evictCached() instead.

Implements Store::Storage.

Definition at line 931 of file MemStore.cc.

References Ipc::StoreMap::freeEntryByKey(), and map.

Referenced by evictCached().

◆ get()

StoreEntry * MemStore::get ( const cache_key * )

overridevirtual

Returns: a possibly unlocked/unregistered stored entry with key (or nil) The returned entry might not match the caller's Store ID or method. The caller must abandon()/release() the entry or register it with Root(). This method must not trigger slow I/O operations (e.g., disk swap in).

Implements Store::Controlled.

Definition at line 312 of file MemStore.cc.

References anchorEntry(), copyFromShm(), StoreEntry::createMemObject(), CurrentException(), DBG_IMPORTANT, debugs, destroyStoreEntry, Debug::Extra(), Ipc::StoreMap::freeEntry(), map, and Ipc::StoreMap::openForReading().

◆ getStats()

void MemStore::getStats ( StoreInfoStats & stats ) const

overridevirtual

Implements Store::Storage.

Definition at line 205 of file MemStore.cc.

References Ipc::Mem::PageId::cachePage, StoreInfoStats::Part::capacity, StoreInfoStats::Part::count, currentCount(), KidIdentifier, StoreInfoStats::mem, Ipc::Mem::PageLevel(), Ipc::Mem::PageLimit(), Ipc::Mem::PageSize(), StoreInfoStats::Mem::shared, StoreInfoStats::Part::size, and UsingSmp().

◆ init()

void MemStore::init ( )

overridevirtual

Start preparing the store for use. To check readiness, callers should use readable() and writable() methods.

Implements Store::Storage.

Definition at line 174 of file MemStore.cc.

References SquidConfig::cacheSwap, Ipc::StoreMap::cleaner, Config, DBG_IMPORTANT, debugs, EntryLimit(), extras, ExtrasLabel, freeSlots, map, MapLabel, maxObjectSize(), Store::Controller::maxObjectSize(), Must, Store::DiskConfig::n_configured, Store::Root(), shm_old, and SpaceLabel.

Referenced by Store::Controller::init().

◆ keepInLocalMemory()

bool MemStore::keepInLocalMemory ( const StoreEntry & e ) const

◆ maintain()

void MemStore::maintain ( )

overridevirtual

Implements Store::Storage.

Definition at line 264 of file MemStore.cc.

◆ maxObjectSize()

int64_t MemStore::maxObjectSize ( ) const

overridevirtual

Implements Store::Storage.

Definition at line 294 of file MemStore.cc.

References Config, SquidConfig::maxInMemObjSize, SquidConfig::memMaxSize, min(), and SquidConfig::Store.

Referenced by copyToShm(), init(), and shouldCache().

◆ maxSize()

uint64_t MemStore::maxSize ( ) const

overridevirtual

The maximum size the store will support in normal use. Inaccuracy is permitted, but may throw estimates for memory etc out of whack.

Implements Store::Storage.

Definition at line 275 of file MemStore.cc.

References Config, and SquidConfig::memMaxSize.

Referenced by stat().

◆ minSize()

uint64_t MemStore::minSize ( ) const

overridevirtual

Implements Store::Storage.

Definition at line 269 of file MemStore.cc.

◆ nextAppendableSlice()

Ipc::StoreMap::Slice & MemStore::nextAppendableSlice	(	const sfileno	fileNo,
		sfileno &	sliceOffset
	)

protected

starts checking with the entry chain slice at a given offset and returns a not-full (but not necessarily empty) slice, updating sliceOffset

Definition at line 747 of file MemStore.cc.

References debugs, extras, map, Must, Ipc::StoreMapSlice::next, Ipc::Mem::PageSize(), reserveSapForWriting(), Ipc::StoreMapSlice::size, Ipc::StoreMapAnchor::start, Ipc::StoreMap::writeableEntry(), and Ipc::StoreMap::writeableSlice().

Referenced by ShmWriter::copyToShm(), and copyToShm().

◆ noteFreeMapSlice()

void MemStore::noteFreeMapSlice ( const Ipc::StoreMapSliceId sliceId )

overrideprotectedvirtual

Implements Ipc::StoreMapCleaner.

Definition at line 833 of file MemStore.cc.

References assert, debugs, extras, freeSlots, Ipc::Mem::PageStack::IdForMemStoreSpace(), Ipc::Mem::PageId::number, MemStore::SlotAndPage::page, Ipc::Mem::PageId::pool, Ipc::Mem::PageStack::push(), Ipc::Mem::PutPage(), MemStore::SlotAndPage::slot, and waitingFor.

◆ pageForSlice()

Ipc::Mem::PageId MemStore::pageForSlice ( Ipc::StoreMapSliceId sliceId )

protected

Definition at line 783 of file MemStore.cc.

References extras, and Must.

Referenced by ShmWriter::copyToShmSlice(), and copyToShmSlice().

◆ reference()

void MemStore::reference ( StoreEntry & e )

overridevirtual

Implements Store::Controlled.

Definition at line 300 of file MemStore.cc.

◆ Requested()

bool MemStore::Requested ( )

static

whether Squid is configured to use a shared memory cache (it may still be disabled due to the implicit minimum entry size limit)

Definition at line 959 of file MemStore.cc.

References Config, SquidConfig::memMaxSize, and SquidConfig::memShared.

Referenced by EntryLimit(), and MemStoreRr::finalizeConfig().

◆ reserveSapForWriting()

sfileno MemStore::reserveSapForWriting ( Ipc::Mem::PageId & page )

protected

Definition at line 794 of file MemStore.cc.

References assert, Ipc::Mem::PageId::cachePage, debugs, Ipc::StoreMap::entryCount(), freeSlots, Ipc::Mem::GetPage(), map, Ipc::Mem::PageId::number, MemStore::SlotAndPage::page, Ipc::Mem::PageStack::pop(), Ipc::StoreMap::prepFreeSlice(), Ipc::StoreMap::purgeOne(), Ipc::Mem::PageStack::push(), Ipc::Mem::PageId::set(), MemStore::SlotAndPage::slot, TexcHere, and waitingFor.

Referenced by nextAppendableSlice().

◆ shouldCache()

bool MemStore::shouldCache ( StoreEntry & e ) const

protected

Definition at line 590 of file MemStore.cc.

References assert, debugs, EBIT_TEST, MemObject::endOffset(), ENTRY_SPECIAL, MemObject::expectedReplySize(), StoreEntry::flags, IN_MEMORY, MemObject::isContiguous(), SBuf::isEmpty(), map, max(), maxObjectSize(), StoreEntry::mem_obj, StoreEntry::mem_status, MemObject::memCache, StoreEntry::memoryCachable(), MemObject::MemCache::offset, Store::Root(), shutting_down, and MemObject::vary_headers.

Referenced by write().

◆ startCaching()

bool MemStore::startCaching ( StoreEntry & e )

protected

Definition at line 657 of file MemStore.cc.

References assert, debugs, MemObject::expectedReplySize(), MemObject::MemCache::index, MemObject::MemCache::io, Store::ioWriting, hash_link::key, map, StoreEntry::mem_obj, MemObject::memCache, StoreEntry::memOutDecision(), Ipc::StoreMap::openForWriting(), Ipc::StoreMapAnchor::set(), and Ipc::StoreMap::startAppending().

Referenced by write().

◆ stat()

void MemStore::stat ( StoreEntry & e ) const

overridevirtual

Output stats to the provided store entry. TODO: make these calls asynchronous

Implements Store::Storage.

Definition at line 226 of file MemStore.cc.

References Ipc::Mem::PageId::cachePage, currentCount(), currentSize(), Math::doublePercent(), Ipc::ReadWriteLockStats::dump(), Ipc::StoreMap::entryLimit(), map, maxSize(), Ipc::Mem::PagesAvailable(), PRId64, Ipc::StoreMap::sliceLimit(), storeAppendPrintf(), and Ipc::StoreMap::updateStats().

◆ sync()

virtual void Store::Storage::sync ( void )

inlinevirtualinherited

Reimplemented in Fs::Ufs::UFSSwapDir, Store::Controller, and Store::Disks.

Definition at line 78 of file Storage.h.

◆ updateAnchored()

bool MemStore::updateAnchored ( StoreEntry & )

overridevirtual

Update a local Transients entry with fresh info from this cache (if any). Return true iff the cache supports Transients entries and the given local Transients entry is now in sync with this storage.

Reimplemented from Store::Controlled.

Definition at line 431 of file MemStore.cc.

References assert, StoreEntry::hasMemStore(), MemObject::MemCache::index, map, StoreEntry::mem_obj, MemObject::memCache, Ipc::StoreMap::readableEntry(), and updateAnchoredWith().

◆ updateAnchoredWith()

bool MemStore::updateAnchoredWith	(	StoreEntry &	entry,
		const sfileno	index,
		const Ipc::StoreMapAnchor &	anchor
	)

protected

Definition at line 445 of file MemStore.cc.

References Ipc::StoreMapAnchor::basics, copyFromShm(), Ipc::StoreMapAnchor::Basics::swap_file_sz, and StoreEntry::swap_file_sz.

Referenced by anchorToCache(), and updateAnchored().

◆ updateHeaders()

void MemStore::updateHeaders ( StoreEntry * )

overridevirtual

Reimplemented from Store::Controlled.

Definition at line 350 of file MemStore.cc.

References Ipc::StoreMap::abortUpdating(), assert, debugs, MemObject::MemCache::index, map, StoreEntry::mem_obj, MemObject::memCache, Ipc::StoreMap::openForUpdating(), and updateHeadersOrThrow().

◆ updateHeadersOrThrow()

void MemStore::updateHeadersOrThrow ( Ipc::StoreMapUpdate & update )

protected

◆ write()

void MemStore::write ( StoreEntry & e )

Definition at line 855 of file MemStore.cc.

References assert, CollapsedForwarding::Broadcast(), completeWriting(), copyToShm(), debugs, disconnect(), MemObject::MemCache::io, Store::ioDone, Store::ioReading, Store::ioUndecided, Store::ioWriting, StoreEntry::mem_obj, MemObject::memCache, StoreEntry::memOutDecision(), shouldCache(), startCaching(), STORE_OK, and StoreEntry::store_status.

Member Data Documentation

◆ extras

Ipc::Mem::Pointer<Extras> MemStore::extras

private

Definition at line 103 of file MemStore.h.

Referenced by copyFromShm(), init(), nextAppendableSlice(), noteFreeMapSlice(), pageForSlice(), and updateHeadersOrThrow().

◆ freeSlots

Ipc::Mem::Pointer<Ipc::Mem::PageStack> MemStore::freeSlots

private

Definition at line 99 of file MemStore.h.

Referenced by init(), noteFreeMapSlice(), and reserveSapForWriting().

◆ lastWritingSlice

sfileno MemStore::lastWritingSlice

private

Definition at line 106 of file MemStore.h.

Referenced by copyToShm(), and copyToShmSlice().

◆ map

MemStoreMap* MemStore::map

private

Definition at line 100 of file MemStore.h.

Referenced by ~MemStore(), anchorToCache(), completeWriting(), copyFromShm(), copyToShm(), currentCount(), disconnect(), evictCached(), evictIfFound(), get(), init(), nextAppendableSlice(), reserveSapForWriting(), shouldCache(), startCaching(), stat(), updateAnchored(), updateHeaders(), and updateHeadersOrThrow().

◆ ShmWriter

friend MemStore::ShmWriter

protected

Definition at line 75 of file MemStore.h.

◆ waitingFor

SlotAndPage MemStore::waitingFor

private

Definition at line 117 of file MemStore.h.

Referenced by noteFreeMapSlice(), and reserveSapForWriting().

The documentation for this class was generated from the following files:

squid/src/MemStore.h
squid/src/MemStore.cc

Classes

Public Member Functions

Static Public Member Functions

Protected Member Functions

Protected Attributes

Private Types

Private Attributes

Detailed Description

Member Typedef Documentation

◆ Extras

Constructor & Destructor Documentation

◆ MemStore()

◆ ~MemStore()

Member Function Documentation

◆ anchorEntry()

◆ anchorToCache()

◆ callback()

◆ completeWriting()

◆ copyFromShm()

◆ copyFromShmSlice()

◆ copyToShm()

◆ copyToShmSlice()

◆ create()

◆ currentCount()

◆ currentSize()

◆ dereference()

◆ disconnect()

◆ Enabled()

◆ EntryLimit()

◆ evictCached()

◆ evictIfFound()

◆ get()

◆ getStats()

◆ init()

◆ keepInLocalMemory()

◆ maintain()

◆ maxObjectSize()

◆ maxSize()

◆ minSize()

◆ nextAppendableSlice()

◆ noteFreeMapSlice()

◆ pageForSlice()

◆ reference()

◆ Requested()

◆ reserveSapForWriting()

◆ shouldCache()

◆ startCaching()

◆ stat()

◆ sync()

◆ updateAnchored()

◆ updateAnchoredWith()

◆ updateHeaders()

◆ updateHeadersOrThrow()

◆ write()

Member Data Documentation

◆ extras

◆ freeSlots

◆ lastWritingSlice

◆ map

◆ ShmWriter

◆ waitingFor