Page 2 of 4
Re: Virtual Grids can view large text files?
Posted: Thu Dec 02, 2021 10:10 am
by edk
edk wrote: ↑Mon Nov 29, 2021 2:32 pm
While working on this task, it turned out that the Grid can be very memory-hungry, especially when we read any property in the QueryData code, be it the Grid control itself, the form window, etc.
To visualize it, just select "Virtual Grid (test memory leaking)" in the "File" menu.
Just try to move the mouse pointer over the Grid and you will see how quickly the available memory is "thinning". Uncheck all three boxes to disable properies polling in QueryData and memory will no longer be consumed at that rate.
I guess generaly there is a problem with a memory leak in the UNICODE version

. In ANSI, there is no leakage in the Grid.

Re: Virtual Grids can view large text files?
Posted: Thu Dec 02, 2021 6:21 pm
by srvet_claudio
edk wrote: ↑Thu Dec 02, 2021 10:10 am
I guess generaly there is a problem with a memory leak in the UNICODE version . In ANSI, there is no leakage in the Grid.
Hi see this post:
https://www.hmgforum.com/viewtopic.php?p=62584#p62584
Re: Virtual Grids can view large text files?
Posted: Fri Dec 03, 2021 12:52 am
by Claudio Ricardo
Dr. Claudio, en su opinión, cual opción conviene más en unicode ?
En un temporizador (en main.prg cada uno o dos minutos) usar:
hb_gcAll (), hb_gcCollectAll () o (de HMG) RELEASE MEMORY, cual de las tres sería más efectiva ?
O antes de salir de la función que demanda mucha memoria usar:
__mvXRelease ( <variable_name> ) ?
Desde yá muy agradecido !
Re: Virtual Grids can view large text files?
Posted: Fri Dec 03, 2021 2:00 am
by AUGE_OHR
hi,
as Claudio say :
in unicode, and is due to the lack of explicit memory release when at c code the strings are transformed into utf8 to utf16 which is the Windows standard
wehn use a "BIG" File (7 GB) you will see that it first grab Size of File but than it become twice Size
when use RELEASE MEMORY i got back 2nd Part of RAM but 1st Part "stay" in Memory until close App
---
now i use Code by Edward to read "BIG" Files.
i (still) wonder that it can handle 7 GB File also under 32 Bit OS
Question : can i "search" in a "virtual" GRID

if Yes how

Re: Virtual Grids can view large text files?
Posted: Fri Dec 03, 2021 2:34 pm
by edk
AUGE_OHR wrote: ↑Fri Dec 03, 2021 2:00 am
Question : can i "search" in a "virtual" GRID

if Yes how
Check this example:
- demo.zip
- (9.66 KiB) Downloaded 167 times
I added the "SeekLine" method and an example of how to search in a classic way (Do While loop) - slower version and using the "SeekLine" method - faster version.
Re: Virtual Grids can view large text files?
Posted: Sun Dec 05, 2021 5:45 am
by AUGE_OHR
hi,
i have a funny Effect with Mouse in that "virtual" GRID
i have add this Code into "QueryFile"
Code: Select all
Function QueryFile( oFile, cDelimiter )
...
IF ! EMPTY(oFile:CurrentLine()) .AND. ! EMPTY(oFile:nLastLine)
Form_1.StatusBar.Item( 1 ) := "Rec. " + STR( oFile:CurrentLine() ) + "/" + STR( oFile:nLastLine )
ENDIF
Return Nil
now click into GRID to "select" any Item and than move Mouse over GRID
you will "see" that Statusbar change ... how is this possible when just "hover"

Re: Virtual Grids can view large text files?
Posted: Mon Dec 06, 2021 5:34 am
by HGAutomator
edk,
Thanks so much for this, sorry I haven't checked the forum for a while.
I'll test this during the week.
Re: Virtual Grids can view large text files?
Posted: Mon Dec 06, 2021 12:35 pm
by edk
Updated version of the SeekLine method (by optimizing the code, the search process accelerated by approx. 50%; bugs related to the record map update were removed):
- demo.zip
- (9.89 KiB) Downloaded 149 times
BTW. Interesting behavior of method and variable in class objects.
First, I read a variable from one object and write its value to a variable in another object.
Then I update the value in the second object, and to my surprise, the value in the first object also changed to take the value from the second object.
I don't know why this is happening, can someone who knows the classes well explain it to me?
You can see this effect by searching for e.g. 11222333 in incremental browsing by calling menu "File" -> Open file (without counting the number of lines) - incremental reading file.
Code snippet showing this situation:
Code: Select all
//I read aRecordsMap from the oFileGrid object and save it to the oFileMT object.
oFileMT:PutRecordsMap ( oFileGrid:GetRecordsMap() ) // <--- If I do not execute this methods before changing aRecordsMap in the oFileMT object, then in the oFileGrid object there is the original value of aRecordsMap.
//Why does this result in "::aRecordsMap" being synchronized between two different objects later in the code?
//aRecordsMap should only be updated in the oFileMT object.
lFound := oFileMT:SeekLine( AllTrim( Form_1.tb_Seek.Value ), Form_1.Grid_1.Value, .T. , { | nPos, nLastPos | Form_1.PBar.Value := nPos / nLastPos * 100 } )
//Why is aRecordsMap the same in both oFileMT and oFileGrid?
MsgInfo ( "Items in 'aRecordsMap' of 'oFileGrid' object: " + Str( Len ( oFileGrid:GetRecordsMap() ) ) + CRLF + ;
"Items in 'aRecordsMap' of 'oFileMT' object: " + Str( Len ( oFileMT:GetRecordsMap() ) ) )
Re: Virtual Grids can view large text files?
Posted: Mon Dec 06, 2021 12:56 pm
by edk
AUGE_OHR wrote: ↑Sun Dec 05, 2021 5:45 am
hi,
i have a funny Effect with Mouse in that "virtual" GRID
i have add this Code into "QueryFile"
Code: Select all
Function QueryFile( oFile, cDelimiter )
...
IF ! EMPTY(oFile:CurrentLine()) .AND. ! EMPTY(oFile:nLastLine)
Form_1.StatusBar.Item( 1 ) := "Rec. " + STR( oFile:CurrentLine() ) + "/" + STR( oFile:nLastLine )
ENDIF
Return Nil
now click into GRID to "select" any Item and than move Mouse over GRID
you will "see" that Statusbar change ... how is this possible when just "hover"
Virtual Grid calls DataQuery even on mouse over.
Maybe try to change "Rec. " + STR( oFile:CurrentLine() ) to "Rec. " + STR( This.value )
Re: Virtual Grids can view large text files?
Posted: Tue Dec 07, 2021 12:07 am
by AUGE_OHR
hi Edward,
edk wrote: ↑Mon Dec 06, 2021 12:56 pm
Virtual Grid calls DataQuery even on mouse over.
Maybe try to change "Rec. " + STR( oFile:CurrentLine() ) to "Rec. " + STR( This.value )
Ok, understand.
STR( This.value ) work fine, Thx