×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Contact US

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

find non vfp file open in a non vfp application

find non vfp file open in a non vfp application

find non vfp file open in a non vfp application

(OP)
In thread 184-1674267 the poster asked how to tell if a file was open. After a bit of prompting he/she explained that it was an Excel file that was "open, meaning visible in your taskbar".

Mike Lewis offered a solution using APIs, GriffMG offered a solution using low level file calls and there was other advice.

I don't know what I'm doing wrong but the low level file calls don't work for me as hopefully is clear from the following:

With the VFP IDE open, use file explorer (or whatever) to open, say "C:\MyFolder\MyFile.txt", a file with some text in it.
Return to the VFP IDE and enter the following (and get the following results)
public gnHandle
public gnBytes
m.gnHandle = FOPEN("C:\MyFolder\MyFile.txt", 12)
?m.gnHandle && 21 (For example)
m.gnBytes = FWRITE(m.gnHandle, "This is the house")
?m.gnBytes && 17
?FCLOSE(m.gnHandle) && .T.

Switch to the open file "C:\MyFolder\MyFile.txt" - there is no change.
Either: close the file without editing it, open it again and there is the "This is the house"
or: edit the file, close and save and then reopen it and there is no "This is the house"

I don't understand why what I get seems to be so much at variance with the advice given, especially from Olaf Doschke, a man with whom I would be very afraid to start an argument. From what I read, I expected the FOPEN() funtion to return -1 if the file "C:\MyFolder\MyFile.txt" was open in another application

Mike Lewis's API solution worked for me, but I also need to know the path name for the file. My knowledge of API usage is very limited, I use them strictly on a "monkey see, monkey do" basis.
However I found 3 possibilities, GetFinalPathNameByHandle, GetFinalPathNameByHandleA and GetFinalPathNameByHandleW. No idea what the difference is. I tried them inside Mike's code. My declaration for all of them was
DECLARE integer GetFinalPathNameByHandle IN WIN32API ;
integer hFile, string lpszFilePath, integer cchFilePath, ;
integer dwFlags
Same for the ...A and the ...W

My usage was
lcBuffer = REPLICATE(CHR(0), 250)
GetFinalPathNameByHandle(hNext, lcBuffer, LEN(lcBuffer), 0x8)
Same for the ...A or ...W. hNext is the handle returned for the file in Mike's code.
This failed. I managed to paste in GetLastError() code from an ancient VFP knowledgebase article. This told me the error was 6 "The handle is invalid"

Could someone please tell me (1) what I'm doing wrong with the low level functions, (2) what I'm doing wrong with the API functions and (3) How to find out if a non VFP file is open in a non VFP application, and if so its path.

If I haven't yet bored you all to death, the practicalities of my question are as follows: My (tiny) application frequently writes to a file named user.log as a means of maintaining an audit trail. The user can view and edit the log from a shellexecute in the application menu. I want to cover the situation where the user (or for that matter anybody else with access to the folder) may have opened the log and perhaps forgotten about it. I want to be sure it is closed in any other application before it is written to by mine. If it is open, perhaps being edited because it has become to big, then tough, out it goes. If there is another application with a user.log and it is open, I want to leave that one alone.

Thanks for your patience

RE: find non vfp file open in a non vfp application

Hi,

You may want to FFLUSH(m.gnHandle) before FCLOSE()

hth

MarK

RE: find non vfp file open in a non vfp application

Quote:

Mike Lewis's API solution worked for me, but I also need to know the path name for the file

Unfortunately, my solution can't help with that. My code doesn't actually look for a file. It looks for a window which includes a specified string within its caption. That string could be a filename, but that is not an intrinsic part of the solution.

Griff's solution might be a better possibility.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

RE: find non vfp file open in a non vfp application

We had a recent thread here...thread184-1820122: How to check if excel is already open when using .workbooks.open

But before you go there, what are the consequences of your experiment with a txt file? You see that notepad does not keep a file open, it reads it in and works with the in-memory copy of it.
Thus changing the file from another process is possible, does not influence the text in notepad, and if you save from notepad, it just overwrites the file, because it also doesn't care that the file was altered in the meantime.

In short: Getting a file handle isn't a mechanism ensuring no application is currently having the file in memory and in its windows controls like a canvas or edit box or whatever for editing of users. It's just ensuring the file is currently not open with a file handle, the low-level key to the file itself.

So it's not a two-way solution. It's that way: If you get no file handle for write mode you are sure something else has a handle in any mode, read or write or both. But if you do get a file handle it says nothing about an editing state. Since an application most of the time works with the loaded file content in memory, usually, even with a representation of an object tree with references, etc. in memory, there is no lock nor a handle on the file that would help you to see the file is worked on.

So in the end you can only be sure about situations where another process keeps the hands on a file and does therefore not let you open it in write or read/write mode. You could usually get a handle for read mode only, but you'll read the file, not the notepad text editor or Excel sheet or any other applications' visual representation in its current state. Only if applications sync the file state with what they display all the time, you would also read that from the file.

In the end, there rarely is anything that works on the hard drive or filesystem-level during the modification you do within an application. An application internally works with the file loaded into memory and some applications may keep a file handle open to ensure no other process can change the file they let its users work on. Other applications, like Notepad, don't care about that. Notepad++ is a bit different from Notepad in that it allows changes from outside, but if it detects a change in a file you currently edit (it does so with a time lag) it will ask you whether you want to reread that file and see its current content.

So, all in all, it depends on how applications are programmed to work with files. In VFP you can also choose how you work on DBFs, with or without manual locks, for example, buffered or not. But even in a non-buffered state, you don't change an FPT memo content with every change of an edit box. The moment the memo is modified in an FPT file is when valid is triggered by setting focus to the next control and the valid event or base behavior of valid returns .t.

A situation where the file is already exactly what you see within an application is as rare as it gets, I assume some hex editors could easily work that way. It's not necessary for the file handle mechanism to work. For that, it would be sufficient if every application would just keep a handle open even if the application doesn't keep the file in the current state displayed in itself, but just to allow such a low-level mechanism to detect this file is in use. But not all applications do that. In VFP you can also use FILETOSTR() and have a copy of a file in a string variable, but there is no file handle kept open you need to close and there is nothing forcing you to finally also do a corresponding STRTOFILE to write changes back.

There is no pairing like that necessary. And that's independent of programming language or file system. It's just a decision of how it's programmed for an application. It could also use a mechanism that's only known to itself and that only other instances of the same application would detect. There is no general low-level OS mechanism that tells you whether a file is "in use" in the sense some application will overwrite it with "save" from its file menu or similar options the user has to decide when to persist modifications in the original file. Editors easily let you save as some new file, which alone is a reason not to act as the current "owner" of a file currently edited.

Chriss

RE: find non vfp file open in a non vfp application

My suggestion relies on the target file actually being open. With Excel and Word this is the case when you open a file
the application leaves it open while you work. If the file is opened in notepad though - it is read entirely and the open
file released. At this point a copy of the file is held in notepad's memory, so my method will not then work...
well, not as expected anyway

Regards

Griff
Keep Smileing

There are 10 kinds of people in the world, those who understand binary and those who don't.

I'm trying to cut down on the use of shrieks (exclamation marks), I'm told they are !good for you.

There is no place like G28 X0 Y0 Z0

RE: find non vfp file open in a non vfp application

Just to be sure exactly what the problem is ...

Are you saying that your application can read and write the log file; that users in other applications can also read and write the file; and you want to be sure that there are no collisions (one set of edits overwriting another) and also that neither application will be prevented from opening the file.

If that's right, I would suggest that a text file is not a good choice. As the others have indicated, there is no way for an arbitrary application to lock a text file.

As an alternative, consider replacing the text file with a DBF. Create a simple, stand-alone log viewer program that does nothing more than let the user view and edit the DBF. Both the log viewer and your main application would then be able to apply all the familiar record locking and exclusive use techniques to prevent collisions and to control lockouts.

Of course, you would have to stipulate to your users that they would always have to use the log viewer whenever they want to access the log. But they might see that as an advantage, given that the viewer could give them more user-friendly features than they would get from working with the raw text.

Just a thought.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

RE: find non vfp file open in a non vfp application

Ah, yes, I forgot to recommend something for the case you want to cover.

Well, the only way I see to let your own application know a log file is in use is that you don't just use a shellexecute and let notepad display that text file, but let that be a viewer application by yourself. Then you can open it "exclusive" by using fopen in mode 12. And then no other application will be able to use the log file until that viewer closes it again.

So you can stick to a txt file, you just have to have your hands on all access to it in a way that a non-wanted change is communicated more or less direct. In case your viewer has the log open in mode 12 there is no chance for any other process to get file access. But you can't know if some user uses notepad to look at the log file and you could only prevent any writing over of new log entries you added while someone edited it in notepad and finally overwrites it, if you keep hands on the file all the time. You usually don't.

The problem of a log file getting too long is usually to only let it grow to a maximum size and then archive that and start a new log file. That way you would remove the need to open the file to empty it. Old archived logs then can be deleted, or whatever suits you. Backed up, zipped, whatever. That's how I would tackle that part of the problem.

What would suit you, I guess is a pause mode, every logging stops or is redirected while a log file is inspected. And after the inspection including perhaps some edits finishes, what would have been written to the logs initial state will then be appended to it.

With your own viewer you could of course do that: Rename user.log to paused.log for editing. Create an empty user.log the main application then uses, let users edit paused.log and at the end of the editing save to paused.log and append the current user.log to the edited content and you have those new log entries "merged" with the log edit. In this case the merge is just an append, as all entries in the newly started empty user.log are after anything in the user.log you renamed to paused.log.

In short, that's what a snapshot mechanism does, at least one way of doing such a thing. You can't instantly copy the user.log to a paused.log, but you can risc the short interval a user.log is renamed to paused.log and no new empty user.log file exists. Even more so, if you program your logging in a way that it adds to a user.log and due to a missing file it creates a new one.

The joining of what was added to the user.log after the editing is a step you surely can't automate if you keep using ShellExecute or notepad for the log editing, that's a step you need to program into the save mechanism of your own log viewer/editor.

Chriss

RE: find non vfp file open in a non vfp application

There are tons of other ideas, I'm sure you can come up with one of them yourself. Don't lock into a problem that's not the actual issue you want to solve, just think about other means. Another simple one would be you could put your application in a mode it would log into memory instead of a file and when you switch back to normal mode it would add what it logged into memory to the file log. That way you simmply avoid the need for the user.log to be available all the time for the application logging.

Chriss

RE: find non vfp file open in a non vfp application

And last not least don't underestimate the simplicity of Mike Lewis recommendation to base the log file on a dbf. Yes, that makes viewing harder, you now actually need a viewer that shows the log, in the simplest case the dbf has a 254 char field and you show it with a browse or a grid. Editing paragraphs, selecting rows for deletion etc. then is a challange to provide, but you can easily look at a dbf while the main logging feature adds new records to it.

Chriss

RE: find non vfp file open in a non vfp application

With a dbf log, if you want to cut and paste, simply write it out to a text file and read that.

Regards

Griff
Keep Smileing

There are 10 kinds of people in the world, those who understand binary and those who don't.

I'm trying to cut down on the use of shrieks (exclamation marks), I'm told they are !good for you.

There is no place like G28 X0 Y0 Z0

RE: find non vfp file open in a non vfp application

Griff, of course selection copy & paste is easier in a text editor, you can also concatenate the log records to a single string and bind an editbox to it instead of displaying the rows in a grid or browse no need to go to a text file first.

Anyway, a "cut" operation is not simple in a dbf, because of the nature of deleted rows just having a deletion mark.

I also don't know what size we're talking off, a log that has a GB size is not easy to handle in any way

But as far as I understand the need to cut rows mainly is to cut off the oldest and first log entries to make room, and that problem is clearly much easier to tackle with a segmentation like only allowing a log to grow to 10MB and then create a new user1.log,user2.log, etc., perhaps even cyclic so that in case you reach user9.log your next log is user1.log deleting that file. Depends, how old log entries you really need. You could also write seperate logs for messages, warnings, errors and flag records with a severity level or any other means to categorize them.

Chriss

RE: find non vfp file open in a non vfp application

I've just remembered that I actually wrote a log viewer in VFP about 10 years ago. The application recorded all its inserts, updates and deletes into a log file, which was a DBF. In the viewer program, the user could drill down by user, date and transaction type. The results were then shown in a grid. The user could select which grid columns to show, change the order of the columns, and make similar adjustments.

I think you will agree that this gives the user a lot more functionality than simply viewing the log as a text file.

But it would have been easy to also provide an export feature, to allow the user to copy some or all of the data to a text file, or a workbook, or whatever.

The 2GB limit was an issue. To solve that, the application periodically checked the size of the DBF. As it approached, say, 1.8 GB, it would delete about 0.5 GB of the oldest records, and then pack the file. This was done during the application's routine monthly maintenance, which permitted exclusive use.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

RE: find non vfp file open in a non vfp application

I don't know how urgent or not the problem is for you, but in summary: Your idea of detecting whether a file is open in any application clearly fails.

Because, in short: A file can be loaded into some application without the file being open with a handle on it, not even a read handle.

So solve your overall problem by logging differently in the first place or providing the full lifecycle management of logging by not only writing the logs, but also offering a log viewer and editor. It's quite straight forward to rather rethink the solution from the bird perspective than being stuck in a detail you can't figure out to solve, just because you think it's just the last little detail you need to solve.

Ask yourself what you would do, if you could detect a file is open in the sense you want to know it for: So you know the application could overwrite it as the save step of its editing. What are your means against this save step? Unless you keep the log file open exclusively and thus already prevent the situation to come up, you can't prevent your new log entries are overwritten by the editing of the log. So even if you solve your detection problem, you're still far away from a solution. aren't you?

Contradict me and tell me what you'd do, if you knew notepad has the user.log file open. What then?
Edit: Notice even if you get no write handle on a file and know it is open right now according to the filesystem, you still don't know which process has the handle that hinders you to get write access to the file.
End of edit.

Do you then know you need to wait for further logging until that editing ends? And what if the user editing leaves the log open and forgets about that? Even until shutting down the workstation? You have to log somewhere else to prevent all current log entries never being persisted at all, don't you?

I do understand you explicitly want to enable editing, so just preventing that is no solution, but you have to have your hands on the full lifecycle of the log, so a log viewer is a natural answer.

There lurk other problems: Files could always be deleted or moved for any reason, by antivirus software, for example, into quarantine. So if a log is more important to you, you'd rather go for using data files for logging or even Windows event log. By the way, Windows offers an application called... Event Viewer (*tada*). Why do you think it does? And why is the Windows Event Log not simply a txt file with all its vulnerabilities?

Chriss

RE: find non vfp file open in a non vfp application

AliMac,

It's nearly a week since you posted your question. You've had around a dozen replies since then, with lots of advice.

It would be really helpful if you now gave us some feedback on those replies. Were any of them helpful? Were our assumptions correct? Was our advice on the right track?

If nothing else, it would be a courtesy to acknowledge the help you have been given. A simple "thank you" would be enough.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

RE: find non vfp file open in a non vfp application

There is one last simple thing to answer we owe to your initial post. You said you get the error 6 from GetLastError, which you looked up translates to:

Quote (AlliMac)

"The handle is invalid"
That's the correct translation.

You can know this in advance. If FOPEN or also if FCREATE returns a negative value, that's not a valid file handle. So before you assume you have opened (or created) a file, you have to check if the handle you get is positive. If you get a negative value back the FOPEN failed because something else has the file open and you then don't need to and can't FCLOSE it.

I can also tell you when you get this. We already said notepad reads in a file and then edits what is loaded in memory. If the log file is big enough, the reading phase takes a while. The more frequently you log, the more probable your logging overlaps with that reading in of notepad and you get no write handle during that time. STRTOFILE would also fail in that case. Both won't trigger an error event, though. FOPEN returns -1 and STRTOFILE would return 0 as the number of bytes written.

When you finally get a valid handle, that's not because notepad has finished editing, though. that's when your logging succeeds again, but will be overwritten, if the user saves his edited notepad text.

As I already said, it doesn't help you to get a handle to know if what you write to the log won't be overwritten.

does it all become clearer now? I can also draw you a timeline of events:


1. User opens user log manually or by your application via SHELLEXECUTE user.log. Anyway, it opens in notepad
|
| notepad load progresses
|
V
| 2. FOPEN: returns <0 value
| 3. FRWITE fails with a negative handle, you don't have a valid file handle, your app crashes or continues ignoring the error
|
| notepad load progresses
|
V
4. Notepad finished loading
|
|
V
|
| 5. FOPEN, FWRITE, FCLOSE succeeds, you log new lines
|
V
6. user edits log file in notepad
|
| 7. FOPEN, FWRITE, FCLOSE succeeds, you log new lines,
| 6.+7. can repeat many times.
|
V
8. user edits in notepad
|
|
V
9. user saves and overwrites your log entries
|
| notepad overwrites the file.
|
| 10. FOPEN gives handle<0, FWRITE fails. That can repeat many times, again.
|
|
V
11. notepad saved

After that logging continues normally.

You lost not just 2, but as many log entries as were made from the point notepad has loaded fully and the user saved. And you lose the log entries during the save period, too. So you will not only have the edited log file but lost log entries that would have been done during the editing phase.

The longer the user edits - and in the worst case he forgets that and does other things the whole afternoon, for example - all that time the log entries you make will be overwritten, when the user finally saves in notepad.

Maybe now you understand why both me and Mike suggest you solve that by providing a log viewer/editor yourself. So you know the state of the log all the time. See ideas in previous posts.

You can't say by having a valid file handle, that the file is now in your hands, fully. It is temporarily in your hands, but notepad can also have it loaded currently, and when the user saves, that overwrites all you write to it during his editing.

The reason a DBF works so nicely is that you can INSERT (or APPEND BLANK & REPLACE) new records while one or even multiple users look at the DBF records in a grid and edit it. All the necessary shared file handling is normal for a DBF file. Then you can provide a feature to delete all entries previous to some datetime. Or select records with a specific word in them, or whatever else users and you yourself need.

Last not least, even now you know how the FOPEN function works (it's all documented what the return value means not only in the normal case when it's the file handle, also when it's negative!), you still don't have a way of full control over the log, if you let users edit it with notepad.

When the log file is small, the time between point 1 and 4 is short, likewise the time between point 9 and 11. Then you still have the problem of all the time between 4 and 9 the user edits while you add log entries, that finally are lost.

Chriss

PS: I wrote a short prg that writes a large file with 1MB size spaces and then adds 20 lines per second with FOPEN,FSEEK,FWRITE,FCLOSE. FOPEN returned a good handle also when I opened the file in notepad, even though it took some time to load as it was initialized to 1MB size. But I got a failed FOPEN when I saved an edited file from notepad. I would have expected more fails of FOPEN, but VFP seems to wait, if it can't immediately get a handle,, that would explain the error tolerance. Nevertheless you get into situatons you don't get a valid file handle, more likely during saving of the edited log, I guess, as you need a write handle that's posessed by notepad in that case and is more problematic than one process only reading while the other wants to write.

RE: find non vfp file open in a non vfp application

Last response to

Quote (AlliMac)

If there is another application with a user.log and it is open, I want to leave that one alone.

The solution to that is have a separate user.log, unless you're having a central log for all clients. Well, that again is better to handle with a central DBF, that's VFPs thing to persist data, isn't it? With all it's ways to use tables shared and handle concurrency and conflicts, transactions and/or buffering and/or locking.

The big advantage of a txt or log file with just Ansi text is the editing with any editor, but the disadvantage more than destroys this advantage.

Chriss

RE: find non vfp file open in a non vfp application

(OP)
Okay, okay I get the message. Using a text file as a log file openable in Notepad is a bad idea.

Thanks for your response mjcmkrsr. I tried fflush() but it doesn't help if another user has the file "open" in Notepad, edits it after the fflush() and then saves his/her edits.

Thanks for your response Mike Lewis. My application writes to the log file and the user can read and write the log file. But of course it's just a text file so anybody with access to the log file's folder could open it also, inadvertently, curiously or even with malicious intent. And yes, I now understand that your code gets a handle to a window not a file, as does some other code I have played with.

Thanks for your responses Chriss Miller. Clearly I didn't know how Notepad (and other applications) work with their "open" files. However it makes it pretty clear I can't rely on getting a file handle to tell me if a file is open in Notepad or not. I had a play after reading your first response and found that I could open a text file in Notepad a number (limitless?) of times by double clicking the file name in File Explorer. They could all be edited and what was actually saved was the editing of the last man standing.

I have been drafting this reply for several days now and keep revising it on getting more responses from Chriss Miller. I appreciate all the time you have put into this Chriss, thanks very much. I certainly have a better understanding of how Notepad works and I can see it (and probably a number of other non-VFP applications as well) is not a good way to log application activity. It all seems to be summed up in your line "Because, in short: A file can be loaded into some application without the file being open with a handle on it, not even a read handle".

Thanks for your response GriffMG. You and Chriss Miller have explained to me how text files and notepad work. I suppose I have never had the urge to open the same text file in Notepad multiple times or I might have worked it out for myself. Never mind, I know now thanks to your responses. I played with the code you posted in thread 184-1674267 and of course I found that although it doesn't work with text files open in Notepad, it works fine for things like spreadsheets and VFP tables. Your cut and paste suggestion needs thinking about too.

So now it's back to the drawing board to come up with a logging system (for errors alos) that is "inside" VFP. Another couple of tables might be on the horizon.

Thanks to all the responders, particularly Chriss Miller. I've saved the entire post to (dare I say it) a text file so I can easily read it over and over again. Thanks also to Tek Tips, a great forum.

RE: find non vfp file open in a non vfp application

Just another point to consider ...

Is there a good reason to allow users to write to a log file? I can understand that they must be able to read it. There would be no point in making a log if people couldn't see it. But if the aim of the file is to log events within the application, why would anyone be allowed to write to it?

This is not a criticism. You know your setup better than me. But it might be a point to keep in mind.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

RE: find non vfp file open in a non vfp application

(OP)
Thanks for your response Mike. The only reason a user would want to edit the user log would be to chop the top off it to bring it down to size. Whether or not this would actually ever be an issue in my tiny application is a moot point. I was really thinking of inadvertent editing or access.

This thread has given me a lot to think about. I thought I was being very clever using a text file and shellexecute for the user log (and the error log and a couple of others). No form or table required, very simple. However I can now see that the disadvantages outweigh the advantages.

So far my thoughts are a table maybe with one field since all the entries generated by the application at the moment are in date and time order anyway, maybe updated on the fly, maybe batch updated when the user logs off. If the latter does it matter if the application crashes and the batch update doesn't happen? Perhaps make an entry on logon so at least there is a record of the crash (no logoff). Probably add a bit of code to delete the first nnn records if the reccount() is more than nnnnn. Lots of ideas suggested by the thread and prompting me to come up with some ideas of my own.

As a final thought, my application has been using text files for logging errors and activity ever since I first created it twenty years ago. I can't remember when I implemented attempting to close a log file that might be open by another user, but I recently discovered that due to a coding error, it never worked anyway. This proves at least that my testing was faulty but it might also suggest that nobody has ever been interested in the log files anyway. However I will persevere if only to keep my brain going.

Thanks everybody.

Thanks everybody who responded.

RE: find non vfp file open in a non vfp application

Regarding "chopping off the top to bring it to size", this is something that can, and should, be done within the application. If you decide to use a DBF, then your program could check the size periodically. As it approaches, say, 1.8 GB, it should delete the oldest. say, 0.5 GB of records, and then do a Pack. You would have to do this when no other part of the system can access the log, as it requires exclusive use.

This is what I do in the application I mentioned above, and it has been working well for over 20 years.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

RE: find non vfp file open in a non vfp application

Txt logging is not at all unusual, the most used VFP function for it is STRTOFILE, not FOPEN/FRWITE/FPUTS/FCLOSE. It has the same problems, though, with user editing of logs.

You say you (and your users) main intent is

Quote (AlliMac)

to chop the top off it to bring it down to size
Why not do as I suggested early on? You can determine the size of the log file and if it exceeds say 1MB (not even 1GB) create a new one with a serial number. Old ones could be archived or erased. That makes editing for that reason completely unnecessary.

And you also said

Quote (AlliMac)

I was really thinking of inadvertent editing or access.
Well, is it inadvertent, if you actually offer the feature to open it by shellExecute from your application? How does it make that inadvertant?
Usually you log errors in such a log so you as developer can look at it, the user has a) a messagebox of an error, and b) usually no idea what to make from the error nor from a list of errors in a log.
There are other logs, maybe you log when the application start and ends, so a mismatch of a start without an end log row means a crash of the application. Such things are quite usual.

The more meaning your log entries have, the more likely you rather want a data file to store what you log and more than just a text field to store the lines of text, to be able to filter better. That a dbf physically in chronological order does only mean that, not that you easily can filter for all records from June 2022, that's simple with a date field, though.

You can also use a dbf you actually initialize with empty records up to the maximum size you want to allow it to grow to and then sort it by a date field. When adding a new log entry you replace data in the record at the top in datetime order, which is the oldest record. Important is you replace the date fielld that sorts it to the current datetime and thus make it the lat record. Keep that order by a datetime filed and you always replace the oldest record with the latest log data and have a fixed size log, where the chronological end is somewhere in the DBF, not at the physical end of file.

Thare are so many ideas, you just have to thnk a bit harder about your wanted features and then work towards them.

Chriss

RE: find non vfp file open in a non vfp application

(OP)
Thanks Mike for your latest post. When I change the log to a dbf I certainly intend to monitor its size within the application. I already have two ideas for that to think about, yours and Chriss's (see below)

Thanks Chriss for your latest response. If I were to stick with a text file for logging, your suggestion is certainly a much better way of limiting the size of the file (if in fact it ever got to be big enough to be a problem). However you have more than convinced me that a dbf is a better solution for logging.

By inadvertent I mean the user might open the log to see what he/she did on such and such a date or on what application record, forget about it and do some editing which results in a log entry. Perhaps inadvertent was not the right word. I also take your point about date and time. Changing to a dbf requires a lot more thinking and I'll be reading this post over and over again to help me.

Thanks again

RE: find non vfp file open in a non vfp application

Quote:

the user might open the log to see what he/she did on such and such a date or on what application record, forget about it and do some editing

It's hard to imagine a situation where the user is able to edit a log file. Surely, the purpose of a log file is to provide a record of the things that actually happen within the application. Why should a user be allowed to change that record?

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

RE: find non vfp file open in a non vfp application

Footprints in the digital snow...

Regards

Griff
Keep Smileing

There are 10 kinds of people in the world, those who understand binary and those who don't.

I'm trying to cut down on the use of shrieks (exclamation marks), I'm told they are !good for you.

There is no place like G28 X0 Y0 Z0

RE: find non vfp file open in a non vfp application

By the way, logs can by error contain information you wouldn't want to leak, like credentials. Such things already have made the news, and not just once. As the major point was shortening the log, that's better made a task of your logging mechanism, and if it comes to important log information that can help you prove what has been done, legal rights of users like data privacy can play a role on the other side, i.e. some companies would get in trouble if you log something identifying the user and the time he did something. Even though on the other side there are tools in which employees are asked to report what they worked on in general.

On the other side log manipulation to change information after the fact can also bring you into trouble. And thus there are techniques that start with computing a simple hash for each log entry (which is not very safe, as it can be recomputed after changing logged information) and go up to cryptographically chaining the log entries, which makes detectable any manipulation. While the information that's removed then still is removed and you only have evidence of manipulation, you can, of course, put log data within a database that limits privileges to who can access it or log in two different locations, etc.

Then we don't talk of DBFs anymore, though a DBF file also can be secured by the file access control mechanisms of Windows/NTFS. The not-so-trivial part is that your application runs with the user's account and privileges and should allow writing the log under that condition, but there are ways to let your EXE impersonate another user - overall or just while logging - and there are ways with Windows cryptographic API that gives privileges to your process, not a user. That makes access privileges only coincide with the user as long as he uses your EXE, not as the user himself when he uses other Windows applications, tools - even hacker tools.

I won't go into details about all of this, but any intention of letting users get to the data they legally are allowed to know without enabling them to manipulate that is possible. I.e. all legal rights of both them and you can be fulfilled, if you just dig deep enough into it.

Just saying this, so you know this is still not at all the end of all thoughts you can put into anything, be it as simple as logging.

Chriss

RE: find non vfp file open in a non vfp application

(OP)
Thanks Mike Lewis and Chriss for your latest responses. The whole logging thing clearly requires a total re-think and it will get that. Very interested in Chriss's remarks on data security and access. My application is probably to insignificant for this to be an issue but as I revise the logging system in general this whole thread will be my guide.

Thanks to everyone for your responses

RE: find non vfp file open in a non vfp application

Thanks for that feedback.

In your case, if you don't need to worry much about the security of the log information - just want no errors during logging - the simplest solution would be to do as VFP does in coverage logging and keep an exclusive file handle all the time. So you get it at application start and don't close it before application end. It stops any parallel access. And you don't even need to worry about crashes. In that case, Windows will close file handles anyway.

You may provide a log view feature within your application, as it's the only process able to write and also read the log file while your application runs.

Of course, log access still is possible after your EXE quits. But if that's okay the exclusive log access during your application session fulfills its job to avoid the problems of concurrency in any of the forms I wrote about. Its intention is not to exclude the user totally from the log, just to avoid any concurrency problems while your application writes to the log.

To go that route of a solution you still could use the file functions fopen/fseek/fwrite/fclose. And, of course, you can use a DBF for logging and can USE yourlog.dbf EXCLUSIVE. The code is even clearer than fopen(file,12) is. Same strategy and reasoning. Including the same feature: In case of an EXE crash this exclusive dBF file access is closed by Windows, too.

Well, and in case USE yourlog.dbf EXCLUSIVE or fopen(logfile,12) fail at application start, you simply refuse to continue and quit.

Chriss

RE: find non vfp file open in a non vfp application

Quote:

And, of course, you can use a DBF for logging and can USE yourlog.dbf EXCLUSIVE.

But what if you have multiple applications all using the same log file? Or multiple users of the same application? In that case, wouldn't it be better to use the table SHARED, and to apply record locking for the very short time that it takes to write to the file?

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

RE: find non vfp file open in a non vfp application

Mike, if you want to prevent any problem of concurrency, isn't it the logical consequence to use a file exclusive?
Even if you now have a central log file, which I doubt makes sense for text-based logging, then you can provide one log file per installation, using ID() within the name when storing it in a central share or storing it local.

Anyway, it is not for data you want to share with other users, it's simply the log data of one installation, one application instance, no more, no less, there is no need to share access to it. That was the root of problems with the log file, wasn't it? That opened in notepad, once you save you overwrite what was logged in the time of editing.

Of course, technically you can use a dbf shared and then both write and read shared. You you can make it a dump of log data of all application installations and instances, and hard to later interpret it, unless you add a field for it to filter for. But then I'd rather prefer to union data of multiple logs, if at all, than the need to filter out just the log entries of one installation. Usually, I look into one instance log data to identify a problem with that installation.

In the end, it depends what you like best and is the decision of AlliMac. In case of keeping it a text-based log file, I'd think of exclusive usage by FOPEN(file,12) or FOPEN(file,2) as a nice way to prevent any concurrency problems and suppress any parallel usage of the file with notepad or any other text editor. In the case of DBF the parallel use of text editors falls flat, that's true, so you can decide to use a shared log. But to see log data per instance separate log files or DBFs are still useful, they also grow less than if storing all log data into one dbf. If you want to do any overall statistics or find common problems in all data, you could append all DBFs into one for that processing.

Chriss

RE: find non vfp file open in a non vfp application

(OP)
Thanks for the latest posts Mike Lewis and Chriss. More food for thought. At present each user has his/her own user log, but when this thread started me thinking more deeply, I wondered if there shouldn't be one user log for the application. After all it is there primarily as an audit trail (although I'm not sure how often it has been used that way), eg did somebody actually send that job sheet by email to a customer who is now claiming he never received it? It might not matter so much who should have sent it, only if it was in fact sent.

However Chriss's latest post suggests a reason for not having a single user log, so back to the drawing board on that, although it's probably not an issue for my application.

Also I guess you could restrict viewing access to the manager or whoever. But my application doesn't have levels of user so that wouldn't work well. Possibly the easiest way of dealing with it in my application would be to have a common log viewable but not editable by all users but with the application in single user mode.

So far the most important issue seems to be control of the log or logs by the application and it seems that the best way of doing that is to keep them in tables. More thinking required for all the rest.

Thanks again

RE: find non vfp file open in a non vfp application

Quote (AlliMac)

It might not matter so much who should have sent it, only if it was in fact sent.

That is speaking for a single shared log. the only reason still speaking against a shared DBF is when you absolutely want to have full control and log without any interference from editing, inadvertent or not. The exclusive access prevents that, but also prevents shared logging.

If you don't fear users trying to look into a DBF with any tool that might not open it shared only, then you may use a shared central DBF, Mike is right about that. At least a DBF does not incentivize users to edit the file with notepad like a txt or log file does. Simply because the file extension .dbf will only be associated with FoxPro on a developer PC, it has no association on a normal user PC, even with a Foxpro application and runtimes installed.

A step beyond that would be storing log data into a SQL Server database table, that's when users will be cut off of finding log data in a single file they can easily get at. It's over the top to set up an MS SQL Server, even just the Express version of it, just to have a log table in there. Too much effort also because in that specific case there is more direct evidence in the outbox of a mail software than in log data. No doubt a log can be your starting point of researching what was done, not only in the case of mails. And for that something users can't manipulate to fake to have done something or hide an error by deleting log entries may be worth using a database server. It's also not that far fetched, if there is a database server available anyway.

Chriss

RE: find non vfp file open in a non vfp application

(OP)
Thanks for your latest response Chriss. I can see that in many applications access to (any) data would need serious control. But my tiny application doesn't warrant that depth. Anyone who uses it (it is supposed to be multi user but I'm not even sure that there are multiple users) is equally interested in data entered by all users and can also edit data entered by other users (such changes would be recorded in the user log of course).

From what I've learned from this thread, a fixed length dbf, viewable but not editable, is probably a much better way of maintaining a user log than my text file method. But this has become a long thread and there is a lot in it to think about. I'll be reading it many times yet.

Thanks everyone.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members! Already a Member? Login


Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close