org.apache.spark.storage

Class DiskStore

Object

org.apache.spark.storage.BlockStore

org.apache.spark.storage.DiskStore

All Implemented Interfaces:

Logging

public class DiskStore
extends BlockStore
implements Logging

Stores BlockManager blocks on disk.

Constructor Summary

Constructors

Constructor and Description
DiskStore(BlockManager blockManager, DiskBlockManager diskManager)

Method Summary

Methods

Modifier and Type	Method and Description
boolean	contains(BlockId blockId)
scala.Option<java.nio.ByteBuffer>	getBytes(BlockId blockId)
scala.Option<java.nio.ByteBuffer>	getBytes(FileSegment segment)
long	getSize(BlockId blockId) Return the size of a block in bytes.
scala.Option<scala.collection.Iterator<Object>>	getValues(BlockId blockId)
scala.Option<scala.collection.Iterator<Object>>	getValues(BlockId blockId, Serializer serializer) A version of getValues that allows a custom serializer.
long	minMemoryMapBytes()
PutResult	putArray(BlockId blockId, Object[] values, StorageLevel level, boolean returnValues)
PutResult	putBytes(BlockId blockId, java.nio.ByteBuffer _bytes, StorageLevel level)
PutResult	putIterator(BlockId blockId, scala.collection.Iterator<Object> values, StorageLevel level, boolean returnValues) Put in a block and, possibly, also return its content as either bytes or another Iterator.
boolean	remove(BlockId blockId) Remove a block, if it exists.

Methods inherited from class org.apache.spark.storage.BlockStore

blockManager, clear

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

DiskStore

public DiskStore(BlockManager blockManager,
         DiskBlockManager diskManager)

Method Detail

minMemoryMapBytes

public long minMemoryMapBytes()

getSize

public long getSize(BlockId blockId)

Description copied from class: BlockStore

Return the size of a block in bytes.

Specified by:

getSize in class BlockStore

putBytes

public PutResult putBytes(BlockId blockId,
                 java.nio.ByteBuffer _bytes,
                 StorageLevel level)

Specified by:

putBytes in class BlockStore

putArray

public PutResult putArray(BlockId blockId,
                 Object[] values,
                 StorageLevel level,
                 boolean returnValues)

Specified by:

putArray in class BlockStore

putIterator

public PutResult putIterator(BlockId blockId,
                    scala.collection.Iterator<Object> values,
                    StorageLevel level,
                    boolean returnValues)

Description copied from class: BlockStore

Put in a block and, possibly, also return its content as either bytes or another Iterator. This is used to efficiently write the values to multiple locations (e.g. for replication).

Specified by:

putIterator in class BlockStore

Returns:

a PutResult that contains the size of the data, as well as the values put if returnValues is true (if not, the result's data field can be null)

getBytes

public scala.Option<java.nio.ByteBuffer> getBytes(BlockId blockId)

Specified by:

getBytes in class BlockStore

getBytes

public scala.Option<java.nio.ByteBuffer> getBytes(FileSegment segment)

getValues

public scala.Option<scala.collection.Iterator<Object>> getValues(BlockId blockId)

Specified by:

getValues in class BlockStore

getValues

public scala.Option<scala.collection.Iterator<Object>> getValues(BlockId blockId,
                                                        Serializer serializer)

A version of getValues that allows a custom serializer. This is used as part of the shuffle short-circuit code.

remove

public boolean remove(BlockId blockId)

Description copied from class: BlockStore

Remove a block, if it exists.

Specified by:

remove in class BlockStore

Parameters:

blockId - the block to remove.

Returns:

True if the block was found and removed, False otherwise.

contains

public boolean contains(BlockId blockId)

Specified by:

contains in class BlockStore