分享

matlab高级编程之内存管理

 htxu91 2013-07-24

熟悉memory命令

memory - Display memory information

Syntax

memory userview = memory [userview systemview] = memory

Description

memory displays information showing how much memory is available and how much the MATLAB? software is currently using. The information displayed at your computer screen includes the following items, each of which is described in a section below:

userview = memory returns user-focused information on memory use in structure userview. The information returned in userview includes the following items, each of which is described in a section below:

[userview systemview] = memory returns both user- and system-focused information on memory use in structures userview and systemview, respectively. The userview structure is described in the command syntax above. The information returned in systemview includes the following items, each of which is described in a section below:

Output

Each of the sections below describes a value that is displayed or returned by the memory function.

Maximum Possible Array

Maximum Possible Array is the size of the largest contiguous free memory block. As such, it is an upper bound on the largest single array MATLAB can create at this time.

MATLAB derives this number from the smaller of the following two values:

  • The largest contiguous memory block found in the MATLAB virtual address space

  • The total available system memory

To see how many array elements this number represents, divide by the number of bytes in the array class. For example, for a double array, divide by 8. The actual number of elements MATLAB can create is always fewer than this number.

When you enter the memory command without assigning its output, MATLAB displays this information as a string. When you do assign the output, MATLAB returns the information in a structure field. See the table below.

Command Returned in
memory String labelled Maximum possible array:
user = memory Structure field user.MaxPossibleArrayBytes

All values are double-precision and in units of bytes.

Footnotes

When you enter the memory command without specifying any outputs, MATLAB may also display one of the following footnotes. 32-bit systems show either the first or second footnote; 64-bit systems show only the second footnote:

Limited by contiguous virtual address space available.

There is sufficient system memory to allow mapping of all virtual addresses in the largest available block of the MATLAB process. The maximum amount of total MATLAB virtual address space is either 2 GB or 3 GB, depending on whether the /3GB switch is in effect or not.

Limited by System Memory (physical swap file) available.

There is insufficient system memory to allow mapping of all virtual addresses in the largest available block of the MATLAB process.

Memory Available for All Arrays

Memory Available for All Arrays is the total amount of memory available to hold data. The amount of memory available is guaranteed to be at least as large as this field.

MATLAB derives this number from the smaller of the following two values:

  • The total available MATLAB virtual address space

  • The total available system memory

When you enter the memory command without assigning its output, MATLAB displays this information as a string. When you do assign the output, MATLAB returns the information in a structure field. See the table below.

Command Returned in
memory String labelled Memory available for all arrays:
user = memory Structure field user.MemAvailableAllArrays

Footnotes

When you enter the memory command without specifying any outputs, MATLAB may also display one of the following footnotes. 32-bit systems show either the first or second footnote; 64-bit systems show only the latter footnote:

Limited by virtual address space available.

There is sufficient system memory to allow mapping of all available virtual addresses in the MATLAB process virtual address space to system memory. The maximum amount of total MATLAB virtual address space is either 2 GB or 3 GB, depending on whether the /3GB switch is in effect or not.

Limited by System Memory (physical swap file) available.

There is insufficient system memory to allow mapping of all available virtual addresses in the MATLAB process.

Memory Used By MATLAB

Memory Used By MATLAB is the total amount of system memory reserved for the MATLAB process. It is the sum of the physical memory and potential swap file usage.

When you enter the memory command without assigning its output, MATLAB displays this information as a string. When you do assign the output, MATLAB returns the information in a structure field. See the table below.

Command Returned in
memory String labelled Memory used by MATLAB:
user = memory Structure field user.MemUsedMATLAB

Total Physical Memory (RAM)

Physical Memory (RAM) is the total physical memory (or RAM) in the computer.

When you enter the memory command without assigning its output, MATLAB displays this information as a string. See the table below.

Command Returned in
memory String labelled Physical Memory (RAM):

Virtual Address Space

Virtual Address Space is the amount of available and total virtual memory for the MATLAB process. MATLAB returns the information in two fields of the return structure: Available and Total.

Command Return Value Returned in Structure Field
[user,sys] = memory Available memory sys.VirtualAddressSpace.Available
Total memory sys.VirtualAddressSpace.Total

You can monitor the difference:

VirtualAddressSpace.Total - VirtualAddressSpace.Available

as the Virtual Bytes counter in the Windows?Performance program. (e.g., Windows XP Control Panel/Administrative Tool/Performance program).

System Memory

System Memory is the amount of available system memory on your computer system. This number includes the amount of available physical memory and the amount of available swap file space on the computer running MATLAB. MATLAB returns the information in the SystemMemory field of the return structure.

Command Return Value Returned in Structure Field
[user,sys] = memory Available memory sys.SystemMemory

This is the same as the difference:

limit - total (converted to bytes)

found in the Windows Task Manager: Performance/Commit Charge.

Physical Memory

Physical Memory is the available and total amounts of physical memory (RAM) on the computer running MATLAB. MATLAB returns the information in two fields of the return structure: Available and Total.

Command Value Returned in Structure Field
[user,sys] = memory Available memory sys.PhysicalMemory.Available
Total memory sys.PhysicalMemory.Total

Available physical memory is the same as:

Available (converted to bytes)

found in the Windows Task Manager: Performance/Physical Memory

The total physical memory is the same as

Total (converted to bytes)

found in the Windows Task Manager: Performance/Physical Memory

You can use the amount of available physical memory as a measure of how much data you can access quickly.

Remarks

The memory function is currently available on Microsoft? Windows systems only. Results vary, depending on the computer running MATLAB, the load on that computer, and what MATLAB is doing at the time.

Details on Memory Used By MATLAB

MATLAB computes the value for Memory Used By MATLAB by walking the MATLAB process memory structures and summing all the sections that have physical storage allocated in memory or in the paging file on disk.

Using the Windows Task Manager, you have for the MATLAB.exe image:

Mem Usage < MemUsedMATLAB < Mem Usage   VM Size (converted to bytes)

where both of the following are true:

  • Mem Usage is the working set size in kilobytes.

  • VM Size is the page file usage, or private bytes, in kilobytes.

The working set size is the portion of the MATLAB virtual address space that is currently resident in RAM and can be referenced without a memory page fault. The page file usage gives the portion of the MATLAB virtual address space that requires a backup that doesn't already exist. Another name for page file usage is private bytes. It includes all MATLAB variables and workspaces. Since some of the pages in the page file may also be part of the working set, this sum is an overestimate of MemUseMATLAB. Note that there are virtual pages in the MATLAB process space that already have a backup. For example, code loaded from EXEs and DLLs and memory-mapped files. If any part of those files is in memory when the memory builtin is called, that memory will be counted as part of MemUsedMATLAB.

Reserved Addresses

Reserved addresses are addresses sets aside in the process virtual address space for some specific future use. These reserved addresses reduce the size of MemAvailableAllArrays and can reduce the size of the current or future value of MaxPossibleArrayBytes.

Example 1 — Java Virtual Machine (JVM)

At MATLAB startup, part of the MATLAB virtual address space is reserved by the Java Virtual Machine (JVM) and cannot be used for storing MATLAB arrays.

Example 2 — Standard Windows Heap Manager

MATLAB, by default, uses the standard Windows heap manager except for a set of small preselected allocation sizes. One characteristic of this heap manager is that its behavior depends upon whether the requested allocation is less than or greater than the fixed number of 524,280 bytes. For, example, if you create a sequence of MATLAB arrays, each less then 524,280 bytes, and then clear them all, the MemUsedMATLAB value before and after shows little change, and the MemAvailableAllArrays value is now smaller by the total space allocated.

The result is that, instead of globally freeing the extra memory, the memory becomes reserved. It can only be reused for arrays less than 524,280 bytes. You cannot reclaim this memory for a larger array except by restarting MATLAB.

Examples

Display memory statistics on a 32-bit Windows system:

memory

   Maximum possible array:             677 MB (7.101e 008 bytes) *
   Memory available for all arrays:   1601 MB (1.679e 009 bytes) **
   Memory used by MATLAB:              446 MB (4.681e 008 bytes)
   Physical Memory (RAM):             3327 MB (3.489e 009 bytes)

   *  Limited by contiguous virtual address space available.
   ** Limited by virtual address space available.

Return in the structure userview, information on the largest array MATLAB can create at this time, how much memory is available to hold data, and the amount of memory currently being used by your MATLAB process:

userview = memory

userview = 
   MaxPossibleArrayBytes: 710127616
   MemAvailableAllArrays: 1.6792e 009
           MemUsedMATLAB: 468127744

Assign the output to two structures, user and sys, to obtain the information shown here:

[user sys] = memory;

%  --- Largest array MATLAB can create ---
user.MaxPossibleArrayBytes
ans =
   710127616

%  --- Memory available for data ---
user.MemAvailableAllArrays
ans =
   1.6797e 009

%  --- Memory used by MATLAB process ---
user.MemUsedMATLAB
ans =
   467603456

%  --- Virtual memory for MATLAB process ---
sys.VirtualAddressSpace
ans = 
   Available: 1.6797e 009
       Total: 2.1474e 009

%  --- Physical memory and paging file ---
sys.SystemMemory
ans = 
   Available: 4.4775e 009

%   --- Computer's physical memory ---
sys.PhysicalMemory
ans = 
   Available: 2.3941e 009
       Total: 3.4889e 009

内存问题处理

Product Support

1106 - Memory Management Guide


This Technical Support Guide is intended to help you understand and prevent 'Out of Memory' errors in MATLAB.

Contents


Section 1: Why Do I Get 'Out of Memory' Errors in MATLAB?

Virtual memory refers to the ability of your computer to use your hard drive as if it were the random access memory (RAM) of your computer. This ability is present on all operating systems supported by MATLAB (if you are not familiar with memory management, you may want to read Section 5 of this document). Thus, if you have a lot of hard drive space, it might appear that memory is almost unlimited. So why do people get 'out of memory' errors in MATLAB?

One cause of 'Out of Memory' errors is that your system has in fact run out of heap space to hold all of your variables. This means that there is no unallocated virtual address space on your computer for MATLAB to use, and therefore no new variables can be created.

The second major cause of 'Out of Memory' errors is memory fragmentation. This means that there is memory available, but there is no contiguous piece of memory that is large enough to hold the specified variable. When virtual memory is used and freed during normal operation of MATLAB, memory becomes fragmented. This means that the amount of total free memory may be greater than the amount of contiguous free memory. Since a matrix in MATLAB must be stored in a contiguous block of virtual memory, the largest matrix that can be created at a particular point in time is limited by the amount of contiguous free virtual address space.

32-Bit Architectures

Most computer platforms today are 32-bit architectures, meaning that the length of a pointer or the size of a processor instruction can be at most 32 bits long. This limitation on the sizes of pointers implies that the memory addresses can be a maximum of 32 bits long, which results in a maximum of power(2,32) possible memory addresses. Since modern operating systems are byte addressable, this translates to 4 gigabytes (GB) of addressable memory. Therefore, under 32 bit architectures, the seemingly unlimited virtual memory space is actually limited to approximately 4 GB.

The Windows operating system further reduces the theoretical limit down to 2 GB of virtual memory due to a design decision to reserve the upper 2 GB of address space. On several versions of Windows, this limit can be moved to allow for an extra 1 GB of memory. For more information about this, please refer to Section 2.

Several 32-bit UNIX operating systems also reserve the upper 2 GB of address space, thereby limiting the virtual memory available to 2 GB. If you need to find out exactly how your operating system reserves virtual memory, we suggest you contact the operating system's vendor.

When you launch MATLAB, approximately 0.8 GB of virtual address space is used to load the heap, stack, DLLs, and other operating system services. For Windows, this limits the net available virtual address space to approximately 1.2 GB to store variables and other data needed by MATLAB.

64-Bit Architectures

If you are using a 64-bit operating system in conjunction with a 64-bit version of MATLAB, it is theoretically possible to access approximately power(2,64) bytes of memory. This far exceeds the amount of memory available to any computer on the market today, so the amount of memory that is available to MATLAB will be limited by the amount of memory available on the computer. As this is typically related to the amount of available virtual memory (or swap space), you may want to consult Section 6 of this document. Please note, however, that there is still a limit on the largest single array available in MATLAB. For more information about this topic, please consult Solution 1-1CAT7.

Viewing Memory Usage

If you are on a Windows platform, you can see the memory allocation along with the largest contiguous block of memory available with the following methods.

  • In MATLAB 7.0 (R14) and later versions, use the command:
          feature('memstats')

    This can also return a value containing the number of bytes in the largest available memory block:

          m = feature('memstats')
    In MATLAB 6.5 (R13):
  1. Download dumpmemmex.dll (instructions on how to do this can be found in Solution 1-19IT9).
  2. Place it on your MATLAB path.
  3. Type the following at the MATLAB command prompt:
          dumpmemmex

For information regarding the output of the above commands, see Solution 1-19IT9.

Section 2: How Do I Avoid 'Out Of Memory' Errors in MATLAB?

For information on avoiding 'Out Of Memory' errors, please see Tech Note 1107.

Section 3: How Do I Defragment the MATLAB Workspace Memory?

There are five functions that you can use to free and defragment MATLAB workspace memory. They are listed in the table below. Click on the each link to see the documentation for that command.

  CLEAR  Removes variables from memory
  PACK   Writes existing variables off to disk, and then reloads them contiguously
  QUIT   Exits MATLAB and returns all allocated memory to the system
  SAVE   Selectively stores variables to disk
  LOAD   Reloads a data file saved with the SAVE command

You can use the SAVE and LOAD commands in conjunction with the QUIT command to free memory by:

  • Saving any needed variables with the SAVE command
  • Quitting MATLAB to free all memory allocated to MATLAB
  • Starting a new MATLAB session and loading the saved variables back into the clean MATLAB workspace.

    For additional information on the five commands mentioned above, type

      help FUNCTIONNAME

    in the MATLAB Command Window where FUNCTIONNAME refers to one of the five commands.

    Note: There are some limitations to these commands. MATLAB relies on the C runtime library and operating system calls to allocate and deallocate memory. Depending on the operating system and MATLAB memory manager, these commands may be handled differently. Also, these commands do not affect the Java class objects created by the Java Virtual Machine (JVM) used in recent versions of MATLAB.

    Section 4: How Do I Change the MATLAB Memory Manager?

    MATLAB 7.0 (R14) and later may be started with several different memory managers. Although in general MATLAB will select the most suitable memory manager for a given architecture and operating system, certain applications may benefit from specifying a memory manager other than the default.

    To specify a non default memory manager, You must set an environmental variable (named MATLAB_MEM_MGR), or supply the name of the desired manager to MATLAB as a command line argument (via use of the -memmgr switch). For information about how to start MATLAB with this option, please refer to the documentation for Windows or UNIX, Linux, and Mac operating systems.

    The following memory managers are available:

    • cache -- This is the default on 64-bit platforms. It was the only non debug memory manager for versions of MATLAB prior to version 7.0 (R14). It provides limited integrity checking.
    • compact and fast -- These are useful for large models or MATLAB code that uses many structure or object variables. Compact is the default selection on 32-bit UNIX platforms and Fast is the default on 32 bit Windows. These have little benefit over cache when working with large matricies. Fast is optimized for Windows but is otherwise similar to compact.
    • debug -- This performs memory integrity checking. It is useful for debugging memory problems caused by user-created MEX-files.

    Section 5: How Does an Operating System Manage Memory?

    A computer's memory must accommodate both the operating system's processes and user processes. The following outline describes what happens when a user starts a program such as MATLAB:

    • User starts the program.
    • The operating system creates an address space for the program to run in, which involves setting up appropriate page tables.
    • The program's text and data segments are mapped from the executable file on disk to the virtual address space; the former being mapped as read only, the latter being mapped as read-write.
    • The stack and heap are initialized.
    • Control is transferred to the program.
    • The CPU tries to access the first instruction. Since it is not in memory, a page fault is generated. This causes the operating system to intervene and allocate one or more pages of physical memory and to copy the code from the file, hence the name "Demand Paged Virtual Memory."
    • The CPU then returns control to the program, which continues executing until it encounters another page fault. At this point, step 6 is repeated. If there is no free physical memory, the operating system will have to either discard (if read only) or swap out (write data to swap file if read write) pages before they can be reused. Typically a least recently used (LRU) algorithm is used for deciding which pages get tossed.
    • This pattern continues for the life of the program.

    When a user executes a program, the operating system creates an address space for it to run in. This address space will include the instructions for the program itself, as well as any data it requires. In a system with memory protection, each process is restricted by the operating system to accessing only the memory in its own address space. However, the combined program memory requirements often exceed the system's amount of physical memory installed on a computer. So, modern operating systems use a portion of the hard disk called a swap file to extend the amount of available memory. This technique, called virtual memory, treats physical memory as a cache of the most recently used data. In a virtual memory system, memory is divided into units called pages (A page is typically 4-8 Kb in size). The set of addresses that identify locations in virtual memory is called the virtual address space. Each process is allocated a virtual address space. The virtual address space can range from 0-4 GB on a 32-bit architecture.

    A process's address space contains the set of instructions and data that is mapped into virtual memory when a program is executed. Virtual memory addresses are translated into physical memory addresses through the use of a look-up table, called a page table. In addition to mapping the entire process into virtual memory, a subset of pages is also mapped into physical memory. As each instruction in the process executes, it is either found in physical memory or is not found (called a page fault). When a page fault occurs, the page that is needed must be moved from the hard disk into physical memory before the instruction can be executed. To make room for the new page, the operating system may need to decide which page to move out of physical memory. This is called swapping. A page fault is time consuming because retrieving data from the hard disk is much slower than obtaining it directly from the physical memory. Operating systems attempt to minimize the number of page faults by swapping multiple pages at once. This becomes a trade-off between operating system overhead and the time saved by minimizing page faults, which is affected by the size of each process's working set.

    The operating system takes care of swapping and translating from virtual address space to physical address space. This means that developers have a flat address space at their disposal. With a virtual memory scheme, the amount of memory available is seemingly limited only by the amount of hard drive space. Virtual memory has more memory available than physical memory because it removes the restriction that an entire process must be loaded into memory at one time.

    Section 6: How Do I Set the Swap Space for My Operating System?

    How you set the swap space for your computer depends on the operating system that you are running on.

    • On UNIX, information about swap space can be procured by typing pstat -s at the UNIX command prompt. For detailed information on changing swap space, ask your system administrator.
    • On Linux, swap space can be changed by using the mkswap and swapon commands. For more information on the above commands, type man command_name at the Linux prompt.
    • In Windows, you can set the amount of virtual memory available using the steps outlined below.

    For Windows XP:

    1. Right-click on My Computer icon and select Properties.
    2. Select the Advanced tab
    3. Select the Settings tab under Performance.
    4. Within the Performance Options GUI, select the Advanced tab.
    5. Click on the Change button to increase the virtual memory.
    6. In the Paging File Size section, select Automatic.

    For Windows 2000:

    1. Right-click on My Computer icon and select Properties
    2. Select the Advanced tab and choose Performance Options.
    3. Click the Change button to modify the virtual memory.

    For Windows NT, 95, 98, and Me:

    1. Right-click on My Computer icon and select Properties
    2. Select the Performance tab and click on the Change button to increase the virtual memory.

    Because modern 32-bit machines have physical memory sizes that approach or exceed their virtual address space, a single copy of MATLAB frequently does not see an increase in available memory from an increase in swap space.

    本站是提供个人知识管理的网络存储空间,所有内容均由用户发布,不代表本站观点。请注意甄别内容中的联系方式、诱导购买等信息,谨防诈骗。如发现有害或侵权内容,请点击一键举报。
    转藏 分享 献花(0

    0条评论

    发表

    请遵守用户 评论公约

    类似文章 更多