Using Graphics Card Memory as Swap

[Death Knight]

Hi,
I have 2 GPU, one is onboard 3300 and one 6850...
Unfortunately 6850 has random resets and doesn't work as good as onboard 3300 with same r600g...
So I don't want to use 6800 since developers improve r600g...

My system has 4GB ram and it's not enough some times. I decide to use my 6850's 2GB RAM as a swap device...
I found an article here that explains how could I do that. : http://en.gentoo-wiki.com/wiki/Using...Memory_as_Swap
Actually this is working and safe since I don't run X11 on 6850.
But there is a problem. My GPU is 2GB. But lspci detects its 256 MB.

triQuad:/dev # lspci -vvvv -s  02:00.0
02:00.0 VGA compatible controller: ATI Technologies Inc Barts PRO [ATI Radeon HD 6800 Series] (prog-if 00 [VGA controller])
	Subsystem: PC Partner Limited Device e174
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 44
	Region 0: Memory at d0000000 (64-bit, prefetchable) [size=256M]
	Region 2: Memory at feae0000 (64-bit, non-prefetchable) [size=128K]
	Region 4: I/O ports at d000 [size=256]
	Expansion ROM at feac0000 [disabled] [size=128K]
	Capabilities: [50] Power Management version 3
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0 <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB
	Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
		Address: 00000000fee0f00c  Data: 4181
	Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Kernel driver in use: radeon

modprobe phram phram=VRAM,0xd0400000,252Mi

command works properly but anything beyond that gives error at dmesg.
For example :

modprobe phram phram=VRAM,0xd0400000,254Mi

gives this error at dmesg.

ioremap reserve_memtype failed -16
phram: ioremap failed
phram: `VRAM,0xd0400000,254Mi' invalid for parameter `phram'
resource map sanity check conflict: 0xd0400000 0xe00fffff 0xc8000000 0xdfffffff PCI Bus 0000:00
modprobe:9455 conflicting memory types d0400000-e0100000 write-combining<->uncached-minus
reserve_memtype failed 0xd0400000-0xe0100000, track uncached-minus, req uncached-minus

So how can I use my whole GPU RAM as a swap?

Also I wonder that

triQuad:/dev # hdparm -t mtdblock0
mtdblock0:
 Timing buffered disk reads:   4 MB in  3.07 seconds =   1.30 MB/sec

triQuad:/dev # hdparm -T mtdblock0 
mtdblock0:
 Timing cached reads:   5454 MB in  2.00 seconds = 2728.44 MB/sec

Why buffered disk reads are too slow?

Also it looks like from lspci, ıt says "Kernel driver in use: radeon". Does radeon driver could conflict with phram module? How could I disable radeon for my 6850 but 3300? And if such a thing is really required?
Thanks.

[bridgman]

Stupid edit limit. No wonder my post count is so high.

My system has 4GB ram and it's not enough some times. I decide to use my 6850's 2GB RAM as a swap device...
I found an article here that explains how could I do that. : http://en.gentoo-wiki.com/wiki/Using...Memory_as_Swap
Actually this is working and safe since I don't run X11 on 6850. But there is a problem. My GPU is 2GB. But lspci detects its 256 MB.

The video memory is all directly accessible by the GPU, but the CPU can only access it through the PCI BAR, basically a 256 MB or less window. IIRC the graphics drivers use the 3D engine to move blocks of video memory around so that the CPU can access them through the window when needed (note that it's usually not needed, the whole idea is for the GPU to do all the work in video memory).

It's probably possible to write a ramdisk driver that runs over the graphics kernel driver and uses the GEM/TTM API to access the entire 2GB of video RAM but I haven't seen one written.

[agd5f]

Also CPU reads across the pcie bus are always slow and the default mem/sys clocks on NI chips are really low until the driver loads and programs them properly.

[Death Knight]

Might be someone could make such a ramdisk driver for gallium later. I guess it's easy to do...
Thanks for answers.

[chithanh]

If you are low on memory, you can try zram (formerly ramzswap/compcache) for compressed in-memory swap.
For instructions, see http://code.google.com/p/compcache/

[ssam]

how much power does a 6850 use? my work out a lot cheap to buy some more memory.