×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Contact US

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Project Compiler suddenly stopped including newer files
3

Project Compiler suddenly stopped including newer files

Project Compiler suddenly stopped including newer files

(OP)
Hi Everybody,

I've been developing xBase programs since dBase II in the early 80s and all of a sudden I'm having the craziest problem, the project manager is skipping files on my most critical project and I have no idea why.

The strange this is that this is the same program I've been continuously developing since 1985, and I've compiled newer versions of it at least once a week without a single problem.

Now, when I compile the program, all the newer PRG procedures, and SCX forms are NOT in the EXE file... and are NOT in the Project File.

I'm wondering if anyone else has seen this problem before.

I assumed the project file was corrupt, so I created a new one and just added the main PRG file, expecting it to rebuild the whole project, but it didn't. In fact based on opening the project file directly as a table, the newer project file literally skipped nearly 600 files that otherwise were in the original, and it's basically saying in the error file that the files were not found. All the files are in the same directory as the project file.

I even tried compiling it on an entirely different computer, so it's not a corrupted version of VFP.

I'm starting to wonder if the compiler somehow hit some sort of limit to the number of files in either the directory or the project file itself?

There are over 4400 files in the directory, including about 1400 PRG files, 400 SCX files and a ton of report and other files.

My workaround so far was to add some of the missing files by hand to the project file, but I worry that I won't know which files are missing without either waiting for a customer to say "there's an error saying procedure xys is not found".

I'm thinking of writing a script to look at all the PRG and SCX filenames that are in the director, but not in the project file so I can know which ones to add, but that's not not the best option because there are other prgs and scx files that aren't actually called, so I'd need to start maintaining my own list of files that can be excluded.

This problem really has me confused. I've been developing code for 40 years ands it's like somebody suddenly telling me up is now down, or blue is now red. There's just no reason for the project to think files are missing when they're right in the same directory.

Any thoughts?

RE: Project Compiler suddenly stopped including newer files

Sounds strange, the first thing I'd try is build with the Recompile All Files option.

With compiling on another PC you've eleminated a lot of reasons already, like is a worn out hard drive failing, so the reason isn't VFP, nor Windows but just hardware getting flaky.

I don't know of a limit of project files and since a pjx - as you know and looked into it already - is just a free table the number of records and project size has no practical limit with the 2GB limit of VFP, most records refer to all the files - scx, frx, prgs, whatever, so it's not even limiting the overall project directory size, as long as the built EXE doesn't have 2GB size.

Is there anything special about any file? Are ole public classes invovled and/or ActiveX controls?

The strangest things I experienced once was compiler creating very small EXE sizes and what fixed it for me was starting VFP9.exe itself elevated to compile which not only automatically registers any COM servers but overall went smoother somehow.

Chriss

RE: Project Compiler suddenly stopped including newer files

The most common reason that files don't get pulled into a project is that they're referenced only indirectly, for example, using a macro or name reference. Could that be what's going on here?

Tamar

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


The most common reason that files don't get pulled into a project is that they're referenced only indirectly, for example, using a macro or name reference. Could that be what's going on here?

Tamar
Not in this particular case. These are just new modules, called directly from the main prg file with "Do SomeNewName". The only thing different here is that the procedure itself is new. The time consuming thing is that since it doesn't bring in the SomeNewName, it also doesn't parse that one for other missing files, so I keep adding the new PRG, then compiling and looking at the .ERR file for whatever it skips next. Rinse. Repeat.

The frustrating thing is remembering to check the .err file for missing parts or it actually generates an EXE with those parts missing, and naturally as I test my code in my dev environment, it runs without any problems because in it's uncompiled state, it can execute the PRG directly.

For now, I keep adding the new stuff after I look at the log to the project, but if I ever want to create a fresh project file, I'll need to add at least 600 program files one by one.


Quote:


Sounds strange, the first thing I'd try is build with the Recompile All Files option.
I tried that first, and even creating an empty project with just the main prg. It seems to arbitrarily grab only some of the files and leaving out over 600 files, so I'm sticking to the old project for now and just adding the new stuff as it comes along.


Quote:


With compiling on another PC you've eleminated a lot of reasons already, like is a worn out hard drive failing, so the reason isn't VFP, nor Windows but just hardware getting flaky.

I don't know of a limit of project files and since a pjx - as you know and looked into it already - is just a free table the number of records and project size has no practical limit with the 2GB limit of VFP, most records refer to all the files - scx, frx, prgs, whatever, so it's not even limiting the overall project directory size, as long as the built EXE doesn't have 2GB size.

Is there anything special about any file? Are ole public classes invovled and/or ActiveX controls?
Since VFP hasn't changed since 2009, I kept looking for reasons why Windows wouldn't find the files, even looking for patterns like whether the prg file had capital or lower case letters. I can't find a pattern.

Quote:


The strangest things I experienced once was compiler creating very small EXE sizes and what fixed it for me was starting VFP9.exe itself elevated to compile which not only automatically registers any COM servers but overall went smoother somehow.
Chriss
I've tried changing the permissions on the folder and even copying the files to a FAT32 filesystem and back so it strips any ownership from the files, but I like that idea. Perhaps if I run it as an admin it could eliminate a rights problem entirely.

I'll experiment with a copy in a fresh directory and empty project file to see if it can find everything and report back in case somebody else stumbles on this thread.

PS. I'm not really new to these forums. I just can't find my credentials and the Forgot Password option didn't work.

RE: Project Compiler suddenly stopped including newer files

I can't promise it's a wonder cure, but to see what I mean, create a shortcut to VFP9.exe and look at its properties. In a tab "Shortcut" you find a button "Advanced" and the advanced properties allows making the elevated start the default:



Little remark on the notice text: Running elevated with this option is the deal with UAC to have a less globally unrestricted access to anything when running Windows under the Administrator account. You're asked to confirm the elevation everytime you start VFP9.exe and can only do so as user of the admin group. That justifies the notice Microsoft gives in this screenshot.

Any processes you start from within VFP9.exe don't inherit the elevated state, so it's less risky than running your whole Windows session as an admin. There's still a risk involved as the VFP9.exe runs elevated and you can potentially delete system files and other stuff you normally couldn't simply damage.

So it's still not fully protecting, mainly from yourself, though. A bit caution is good when you run the IDE elevated.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
For the record, I just tried two more fixes, including Chris Miller's suggestion of checking the Run as Administrator box and creating another empty project file and it still leaves out exactly the same 590 files. Meanwhile 590 is just the tip of the iceberg, because many of those missing 590 would've called another 500 or so programs. I also tried re-applying the ownership of the folder to my own account without any change.

So, as it stands I'm still able to build using the older project file, but I literally need to keep adding any new functions, forms, procedures, and even image files that I use for icons after I dig through the error file for what wasn't added.

It's kind of crazy when you can literally see the names of each missing function in the .err file, but every single file is right there in the folder.

What bugs me most is that I've been continuously developing and maintaining this code for nearly 40 years, and this problem only popped up within the last 60 or so days. Since it happens on multiple computers, including Windows 10 and 11, I'm completely puzzled why it suddenly started. I used to just run a quick PRG I called "MakeVovio" to copy everything to a staging folder, generate the EXE from the project, and build a distributable zip file I could deploy without needing to touch the project at all.

At least I know I can keep adding the missing stuff as it comes along, but it's hard to imagine what's going on under the hood.

RE: Project Compiler suddenly stopped including newer files

(OP)
I tried another test by copying everything over to an exFAT USB flash drive, which strips the permissions entirely.

As I was copying the files over, I was encouraged because I saw dozens of popups telling me that I would be losing file attributes, even on files from 2009 and earlier.





I thought for sure this would play some role, but in the end, when I tried to build a new project file, the PJX file had a record count of only 252 rows, but my live project file has over 1400 records and that grows every week or so as I continue to develop new features for my customers. Every attempt to build a new project file always results in only 252 rows in the PJX file, even after copying the whole folder to exFAT, FAT32, and other drives. Spooky.

It actually creates the EXE file, and the ERR file tells me there are 590 errors, for missing functions. This means that if I ever need to build a new project file, I would need to add those 590 files one by one to the project... THEN, it still would be another 600 or so files short, so I'll need to keep manually adding files until all 1400 files are there.

So, as it stands, I'll keep adding files one by one when I see new files pop up in the .err file when I compile, and making sure that if I ever need a new project based on this codebase, I'll just need to copy that older project file.

If anyone else with a large project has had similar problems, I'd love to see if they can identify the cause and share it here.

I'm tempted to build a Windows XP or Windows 7 Virtual PC just to see if it can build my project file from scratch without needing to tinker with it. In fact, that's exactly what I'll try next just to see if this has something to do with some recent Windows updates.

RE: Project Compiler suddenly stopped including newer files


Joe,
you can upload pjt and pjx file for analysing.

mJindrova

RE: Project Compiler suddenly stopped including newer files

Joe,

Did you already PACK the project (it’s a DBF file) before compiling?

Regards, Gerrit

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


Did you already PACK the project (it’s a DBF file) before compiling?
Regards, Gerrit
Yes. Right now, my primary PJX has about 1400 records, but the strangest thing is that if I create a brand new project file and add just the main PRG file, it will only have 252 records after building.

In the past, it would normally parse everything, and I'd have a perfect project file with all 1400 records for everything referenced directly or indirectly from the main prg without having to add any file other than the main PRG to build it.

When starting from an empty project file, the resulting tiny EXE file will report 590 errors in the .ERR file, which means the parser couldn't find hundreds of files that are literally in the same folder. Chances are, even if I added the 590 "missing" files one by one, it would still not parse them and find the remaining files.

Thankfully, I still can use the project file, but I find myself adding anything new I create one by one, which I never needed to do in the past.

Quote:


Joe,
you can upload pjt and pjx file for analysing.
mJindrova

There's nothing unusual in the pjx file. It will do this even if I create a brand new project file and give it the main prg as a starting point.

RE: Project Compiler suddenly stopped including newer files

Did you know you can add files to a project with code?

To me it seems a major part of your function is never directly referenced nor indirectly by macro substitution. Do you use PRGs for functions you then call in other code siumply by PRG name? That does not suggest to the VFP build process that such files should be looked for and added into the pjx, the err file would just report missing functions.

So I think the problem is easily healed if you know you want to include all PRGs in one or more folders, just write a loop to add them in:

CODE

For lnI=1 To ADIR(laPRGs,'c:\some\folder\*.prg')
_vfp.ActiveProject.Files.Add('c:\some\folder\'+laPRGS[lnI,1])
EndFor lnI 

Chriss

RE: Project Compiler suddenly stopped including newer files

A wild guess: Seems to me a last ditch effort in the worst case might be VFP itself or even the operating system, depending on when they were last installed. Were they all installed recently from the same source (possibly defective?)

Steve

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


To me it seems a major part of your function is never directly referenced nor indirectly by macro substitution. Do you use PRGs for functions you then call in other code siumply by PRG name? That does not suggest to the VFP build process that such files should be looked for and added into the pjx, the err file would just report missing functions.
Most of the missing files are PRGs that are simply run via "DO XYZ" without any macro substitution or special handling... Just "DO XYZ", where XYZ.PRG is a regular procedure with just that code.

Normally, if you have a main program that says "DO XYZ", the project builder will include XYZ.PRG automatically. The funny thing is in the main PRG, there are probably 30 "DO" calls to various features. Out of that list, 25 were pulled in, and there is no apparent reason for the other 5 to be missing.

You can tell it definitely parses the other 25 including some of the DO calls and function calls from those, which is how it ultimately ends up with 252 records in the project file... but it's nowhere near the 1400+ in the project file I've used for decades.

At least I've started to monitor the .ERR file and I know when to add files manually, but it's still a mystery. I wrote 100% of this code myself over the past 40 years, with two exceptions, I have XFRX and FoxyPreviewer source code in that folder. Now that I think of it, I recently downloaded newer versions of these, so I wonder if there are any preprocessor commands or issues that would interfere with the project building process. I think I'll tinker with that next. I know my own code builds flawlessly every time.

Quote:


Did you know you can add files to a project with code?

So I think the problem is easily healed if you know you want to include all PRGs in one or more folders, just write a loop to add them in:

CODE

For lnI=1 To ADIR(laPRGs,'c:\some\folder\*.prg')
_vfp.ActiveProject.Files.Add('c:\some\folder\'+laPRGS[lnI,1])
EndFor lnI 

Chriss
I've considered that, or creating a report that just tells me which prgs and scx files are in the folder and not in the PJX, but there are hundreds of other programs in there that are just for testing or other projects, so it would be easier to just keep adding files for now as they pop up in the .err.

The main problem right now is that I'm just plain curious why the project can no longer be built from scratch using the main prg. It's been a while since I've needed to do this, but it worked perfectly in the past, and now it doesn't. The programmer inside of me loves solving puzzles, but this one has me stumped.

Quote:


A wild guess: Seems to me a last ditch effort in the worst case might be VFP itself or even the operating system, depending on when they were last installed. Were they all installed recently from the same source (possibly defective?)

Steve
I tried it on multiple computers using identical copies of the code on flash drives. It's just plain strange. One of the things I love about FoxPro is that it's been the same for years, so it should be predictable. I keep wondering if I stumbled on some sort of size limit, as if it asks Windows for a list of files and it only returns the first 1000 or so. I know it's not a nesting depth limit, because 5 of the missing files are just "DO XYZ" from the main PRG, so it should've included all the DO files first. There's one section that says "if (something)" followed by 5 different Do this, do that, and do theother, and I can see four of them in the project, but not the 5th. There's nothing different in that 5th DO. It's just a PRG just like the rest.

Very very strange.

RE: Project Compiler suddenly stopped including newer files

Notice it can play a role where you CD to (OR SET DEFAULT) when you build. Also if you SET PATH in your development environment.

So it can make a build difference whether you're in the right default folder while building.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
I always start with a CD into the directory with the source, and the path and default folders match the Home Directory that the project uses, which is why the rest of the project files are found without issue.

My next experiment is to remove XFRX and FoxyPreviewer from the folder. Those are the only third party programs, and they both were recently updated to the latest paid versions with source code. Perhaps one of them triggers is confusing the project manager.

RE: Project Compiler suddenly stopped including newer files

Joe,

I can hardly consider XFRX to be causing this. I’ve used many versions for more than 10 years and nothing similar ever happened. I think Martina can confirm this.

Have you checked user rights and file attributes (NOT read only or hidden) for the directories and files you want to have in your project?

Regards, Gerrit

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


I can hardly consider XFRX to be causing this. I’ve used many versions for more than 10 years and nothing similar ever happened. I think Martina can confirm this.
I've used XFRX for quite a few years too, but I'm just looking for things that changed recently other than my own code and I know I renewed XFRX and FoxyPreviewer. In fact, in the .err file, XFRX is one of the countless files it skips.

Meanwhile, all the XFRX files are right there in the same folder as the rest of the source.

Quote:


Have you checked user rights and file attributes (NOT read only or hidden) for the directories and files you want to have in your project?
That was one of the first things I did. As I mentioned, I even copied the whole folder to an exFAT formatted drive which has doesn't have NTFS permissions, and I made sure the entire whole folder wasn't marked read only. It resulted in exactly the same 252 records in the PJX instead of the normal 1400+.

RE: Project Compiler suddenly stopped including newer files

Would you care to run a simple test to check whether there's some kind of misconfiguration or limit?

Think of a new directory, specify it here and run:

CODE

#DEFINE HOMEDIR "C:\buildtest\"
MKDIR (HOMEDIR)
CD (HOMEDIR)
StrToFile("","main.prg",0)
For I = 1 to 2000
StrToFile("do program"+PADL(i,4,'0')+Chr(13)+Chr(10),"main.prg",1)
StrToFile("*","program"+PADL(i,4,'0')+".prg",0)
EndFor
Create Project test.pjx nowait
_vfp.activeproject.Files.add('main.prg') 
and then build it.

The overall main.prg code is much simpler than yours, naturally, and it's all a very flat structure, but I end up with all 2000 PRGs added through the build.

One reason you might have problems is codepages. But before we'd go into that direction, this should not fail with whatever codepage you currently use. Well, and if it works, there are a few less assumptions to worry about, like the number of PRGs limit.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


Would you care to run a simple test to check whether there's some kind of misconfiguration or limit?
...
...
One reason you might have problems is codepages. But before we'd go into that direction, this should not fail with whatever codepage you currently use. Well, and if it works, there are a few less assumptions to worry about, like the number of PRGs limit.

Chriss
I like what you're suggesting and I'll give it a try. I'll also need to modify it to add all the SCX files too.

I'm 100% sure it will compile, and I'm sure nothing will be left out, but it will definitely also include about 300 unrelated or older PRGs that are not part of the real project. If nothing else, perhaps the compiler will suggest I remove them and it's a winner.

The thing that still boggles my mind is how it ended up happening in the first place. Normally, just adding the main program is all you need to build a complete project.

I'm intrigued with the idea that codepages are playing a role. You may be on to something. There was a time where I edited some PRGs using a couple of different editors and search and replace programs like Notepad++ and another one that had file compares for example.

I would use them to simplify refactoring variable and field names. Perhaps one of those programs saved the resulting files in another codepage without me knowing about it and FoxPro is looking for direct matches where it is trying to match a different character for the names of the functions or procedures. I'm going to tinker with that idea by taking the main program and re-saving it into notepad or something that lets me change the codepage, then re-building a new project. It normally can't find about 3 files. If it doesn't skip anything, we've got the answer and I'll just need to see which other files are also in the wrong codepage.

RE: Project Compiler suddenly stopped including newer files

I wonder if you confuse my earlier idea with the last test. This generates 2000 PRG files but only adds main.prg to a new project and then build should add the 2000 PRGs. And that tests whether that mechanism is brokwn for you, doesn't work on your computer or with your VFP version and may find problems

As you mention Notepad++: I'm not sure 100%, but I think by default it would use UTF8 and that has an overlap with ANSI in the first 128 characters, but then when something is special you get problems finally compiling that in VFP. But you already said

Quote (Joe Crescenzi)

looking for patterns like whether the prg file had capital or lower case letters. I can't find a pattern.

If you write á in Notepad++ and save it as UTF8 encoded file VFPs editor would show it as á (at least in ANSI codepage 1252.
It's not the only problem with file contents, the file systems also don't just use ANSI for file names, and even something as simple as starting to use spaces in PRG file names, obviously DO my program.prg would fail and you should use DO "my program.prg" or even more generally DO ("my program.prg"). Because once you have the brackets - aka name expressoins - you can also start using concepts like DO (BASDÈDIR+lcPrefix+"hardcodedpart"+PADL(number,4,'0')) using expressions that combine constants, variables and functions, whatever, if only you get a string finally - in short name expressions.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
I didn't read your script fully, but now that I do, I see exactly what you were thinking. You are creating an entire directory of 2001 program files with just a comment that does nothing, plus one main program that calls all 2000. That's a great experiment.

As I mentioned, I reference XFRX in the main program and that was listed as not found too and I never touched or edited that one.

I keep wondering about the codepage, but then again, it's not just skipping PRG files. It misses class libraries, forms, and even image files like PNG.

Here's an except from the 590 skipped files in the .ERR file. Keep in mind 100% of these files are in the same directory as the main PRG and all 252 files that do get included in the project rebuild. As you can see it's a mixture of just about every type of file.

CODE

Program f:\cartpro.new\cartpro.fxp has the following errors:
    Proc./Func. XFRX2 - Undefined
    Proc./Func. __ID2ID2 - Undefined
    Proc./Func. FIXORPHANS - Undefined
    Proc./Func. CHANGEACCOUNTID - Undefined

Program f:\cartpro.new\tce_lib.prg has the following errors:
    Proc./Func. US_OPEN - Undefined
    Proc./Func. DB_CLOSE - Undefined
    Proc./Func. VL_OPEN - Undefined
    Proc./Func. WDROP - Undefined
    Proc./Func. PAGEBRK - Undefined

Program f:\cartpro.new\logon.prg has the following errors:
    Proc./Func. ASKPW - Undefined

Program f:\cartpro.new\snoop.prg has the following errors:
    Proc./Func. WDROP - Undefined

Program f:\cartpro.new\fleetmenu.prg has the following errors:
    Form FLEETMENU - Undefined
    Proc./Func. FLEETMANAGE - Undefined
    Proc./Func. FLEETBROWSE - Undefined
    Form FLEETDAILYCHECK - Undefined
    Form FLEETMANAGER - Undefined
    Proc./Func. PICKFLEETCODE - Undefined
    Unknown MYGUID - Undefined
    Form FLEETEDIT - Undefined
    Proc./Func. FLEETREPORTS - Undefined

Program f:\cartpro.new\scalemenu.prg has the following errors:
    Form SCALEMENU - Undefined
    Unknown GEN_TSID - Undefined
    Form EDITSCALETICKET-V5 - Undefined
    Form EDITSCALETICKET-BASIC-V4 - Undefined
    Form EDITSCALETICKET-V2-GAETA - Undefined
    Proc./Func. BLANKSCALETICKETS - Undefined
    Unknown PICKSCALETICKET - Undefined
    Form SCALETEMPLATEBROWSER - Undefined
    Form TARESEDITOR - Undefined
    Form FINDSCALETICKET - Undefined
    Proc./Func. REPORT2PRINT - Undefined
    Proc./Func. SCALEREPORTS - Undefined
    Form HAULERMANAGER - Undefined
    Proc./Func. SCALE2PENDING_MANY - Undefined
    Proc./Func. SCALE2INVOICE_MANY - Undefined
    Proc./Func. SCL_STAT - Undefined

Form f:\cartpro.new\routemanager4.scx has the following errors:
    Proc./Func. REPORT2PRINT - Undefined
    Visual Class Library VOVIOMASTERCLASSES - Undefined
    Form ASKROUTEREPORTSTYLE - Undefined
    Proc./Func. REP115 - Undefined
    Unknown DAY2TXT - Undefined
    Unknown NIGHT2TEXT - Undefined
    Form SURVEYCONFIRMPOST - Undefined
    Proc./Func. QSURVEY - Undefined
    Form ASKROUTEEXPORTOPTIONS - Undefined
    Unknown CLEANFN - Undefined
    Unknown L_O_M - Undefined
    Unknown DAYS1E - Undefined
    Visual Class Library SYSTEM2CLASSES - Undefined
    Visual Class Library VOVIOEXTRAS - Undefined
    Form QUICKDISPATCH - Undefined
    File PRINTER-24X24.PNG - Undefined
    Unknown PICKVEH - Undefined
    Form ADDCLIENTTOROUTE - Undefined
    Unknown NEWSORT - Undefined
    Unknown NEWNUM - Undefined
    File G.PNG - Undefined
    Visual Class Library VOVIOBASECLASSES - Undefined 

RE: Project Compiler suddenly stopped including newer files

(OP)
I just ran the test program and it created a test project with exactly 2002 records, as it should.

Still wondering why it only adds 252 records out of over 1400 when I try to build a fresh project from scratch. I'm going to find a way to tinker with the codepage on the main program to see if it picks up any additional files. If I get anything higher than 252, I'll try changing codepages elsewhere too.

RE: Project Compiler suddenly stopped including newer files

What about the pattern that all addressed errors are about the directory cartpro.new?

What happens, if you remove the dot in that directory name and instead call it cartpro_new? And rename all references, of course.
Just checked, this form of directory name does not cause a problem. But, of course, if your new PRGs use old PRGs and address them as if they were in the same directory, that will cause missing procedure/function errors, wouldn't it?

Chriss

RE: Project Compiler suddenly stopped including newer files

By chance: is cartpro.new a zip file and not a directory?

Chriss

RE: Project Compiler suddenly stopped including newer files

Joe,

Another wild guess: do you have enough headroom in the dbf files in the user folder \AppData\Roaming\Microsoft\Visual FoxPro 9.

I recently had an error because the file refdef.dbf had become too large (FPT > 2GB).

You may check the dbf's in this folder to be sure.

Regards, Gerrit

RE: Project Compiler suddenly stopped including newer files

(OP)
The directory is just a complete copy of the newest source code directory. As I mentioned, I'm doing all my testing on a complete copy on the exFAT formatted drive so I can tinker with it in a sandbox, and so that I can rule out any NTFS permissions, because FAT32 and exFAT don't support ownership.

All the files are in the same directory, and there are no references in any of the code to a path.

My last experiment was particularly fun. I just wrote a newmain.prg that had nothing but the same four DO lines that were note included in the real main to see if it would still exclude them. Strangely, it did include them, and even started to parse the references in them, which still eventually led to a long list of different files that are being excluded.

I have to say this is the strangest thing I've ever encountered in my 45+ years of writing code. I'm glad I have a workaround by adding files by hand, but it still doesn't make sense. I should always be able to just create an empty project, add the main prg or scx, then let it rebuild the project. There's nothing different about the PRG, SCX, VSC, or PNG files that are being excluded.

The one thing that somehow still seemed possible is the codepages, but that's proving tricky to confirm. I edited my main.prg using notepad and saved it as ANSI and got the same result. Of the four skipped files in that prg, three of them are called back to back with no code in between. I looked at the codepages in the live and generated projects and everything says 1252, except the kinds of files that have none like vcx, png, etc.

An idea just hit me. I'm not sure of the exact method the project manager uses to add files to the project, but all the files skipped in main are referenced as a DO at the very bottom of main. Perhaps the exact sequence that it adds files is a factor. I noticed that the records in a PJX file are not alphabetic, so the question is what order are they added when building from scratch and how does it play a role, if any.

When looking at the PJX of any of the fresh 252 row versions, it seems to flow like a tree, where the rows flow in something close to the order they were recursively found, with the last rows representing objects found in subsequent files, but clearly not including any of the DO calls made at the tail end of the main, which means it doesn't add all the references in main first, followed by a chain of subsequent parsed files, or those DO calls in main would absolutely be found.

This leads me to the conclusion that somehow there's potentially something that interrupts the parser. I checked row 252, it's just a simple text formatting function with no child references, so it wasn't that one. Perhaps whatever file it would've looked at as #253 stops the normal flow. Finding that will be a challenge, but it's the best guess right now.

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


By chance: is cartpro.new a zip file and not a directory?
Chriss
Just a plain folder.

Quote:


Another wild guess: do you have enough headroom in the dbf files in the user folder \AppData\Roaming\Microsoft\Visual FoxPro 9.

I recently had an error because the file refdef.dbf had become too large (FPT > 2GB).

You may check the dbf's in this folder to be sure.

Regards, Gerrit
My dev machine has a tiny RefDef, about 15,000 rows.

It also does this on a PC with a freshly installed copy of Windows and VFP.

RE: Project Compiler suddenly stopped including newer files

Quote (Joe Crescenzi )

This leads me to the conclusion that somehow there's potentially something that interrupts the parser. I checked row 252, it's just a simple text formatting function with no child references

What do you mean by that? Is there a function definition in main.prg and you have DO calls after that? Those DO calls can never run and thus are not parsed by building, indeed.

The firt line in a prg that defines a procedure, function or class ends the code section before it that has potential code that can run when you DO the PRG, the code in procedures, functions and classes runs when you call them or instanciate a class, but any single command outside a procedure, function or class after such definitions is regarded as non existant by the compiler, how would you reach it, there is no GOTO linenumber in VFP, it's not BASIC.

Chriss

RE: Project Compiler suddenly stopped including newer files

I had the same thing after I included a .h file with #Define instruction in .prgs (with #include command) and forms (through menu).
After I removed these all was back to normal. This problem occurred in several of my projects.
I couldn't reproduce in a simple example so there might be a combination with another thingy.
Question: did you include a file in a form? (Modify form, menu Form, Include file)

RE: Project Compiler suddenly stopped including newer files

(OP)
No. That's not what I meant.

I'm saying that those were just DO calls that coincidentally are just the last DO calls within the file, and are part of the same procedure as those that did come through.

Do calls higher in the main were found. None of the DO calls after a certain point were found, meaning it stopped looking at some point before reaching the end, which also means it doesn't just add all the direct referenced files before parsing subsequent files.

It starts at the top of main, finds a new file, parses that and its subsequent file references, then goes back to main, and repeats with the next (new) object.

If something in the middle somehow fails (for reasons still unknown), it never finishes adding the referenced files at the bottom. If it did, every simple DO within the main would be in the projec first, followed any objects called from those objects, but based on the sequence in the PJX, it looks like it adds them to the project in order found as it branches out.

Example.

CODE

PROCEDURE main
   DO ABC
   DO DEF
   DO GHI
   DO JKL
   DO MNO
ENDPROC 

If the parser just adds objects in main first the project rows would be:

MAIN
ABC
DEF
GHI
JKL
MNO

In that order.

What I'm seeing is more like:
MAIN
ABC
XYZ (a procedure potentially called by ABC)
PDQ (a procedure potentially called by XYZ)
DEF
GHI
(other procedures called by GHI)
... etc.

BUT... NO JKL, or MNO.

So something in the middle must be stopping it from adding anything past a certain point in main. The trick is finding which file is breaking the parsing and understanding why it happens.

As I said, my workaround lets me deliver updates, but my curiosity leads me to know why. To do this, I think I'll keep tinkering with variations. I may even resurrect an old DOS technique using the legendary "FOXBIND" to put all the prgs into one big procedure file and seeing if it compiles. In the old FoxBase / DOS days, I would foxbind all my prgs into one file I called overlay.FP, then compile it as overlay.fxp and use SET PROCEDURE TO OVERLAY at the top of my main. Back then you didn't compile to an EXE, but ran your programs with the runtime at the command prompt from a batch file that said "FOXR MAIN".

This brings back other memories, such as having MFoxPro for the "Multi-User Version".

RE: Project Compiler suddenly stopped including newer files

I see.

Quote (Joe Crescenzi)

The trick is finding which file is breaking the parsing and understanding why it happens.
I agree, that's the bet lead you have.

I'd wonder if you have any double file names, but that would only work with multiple folders of files.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


I had the same thing after I included a .h file with #Define instruction in .prgs and forms.
After I removed these all was back to normal.
I couldn't reproduce in a simple example so there might be another thingy.
Question: did you include a file in a form? (Modify form, menu Form, Include file)
No, but you may have found the answer.

I don't use any .h files, but I recently updated to newer paid versions of FoxyPreviewer and XFRX and I purchased them with source code so I pull in the PRG versions, and as far I know XFRX has some .H files. I'm going to give it a try in a copy of my folder with XFRX source code removed or reference it as an external procedure so it doesn't try looking for it.

RE: Project Compiler suddenly stopped including newer files

I don't see a major advantage of pulling in the FoxyPreviewr source code. I know they suggest that in some circumstances, but I would rather just use the app.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
Just checked. I don't have FoxyPreviewer in source, and it's only called in some test modules. I still use XFRX as my main PDF tool.

I picked up FoxyPreviewer for testing only after learning that they were starting to fix some of its older bugs, especially with 4K and higher screen DPI, which made the old version useless.

That leaves just XFRX, which is definitely being pulled in as a PRG, and that calls some .h files. If the compiler struggles with certain .h files, making it an external reference should do the trick.

RE: Project Compiler suddenly stopped including newer files

(OP)
For the record. I copied everything to a new folder again, then deleted XFRX.* so it couldn't possibly try to find it or any of the .H files, and the only difference is that the PJX has 251 rows instead of 252. The only row that is different is just the lack of XFRX in the project.

The mystery continues.

That said, I'm sure there's at least one module that confuses the builder. I'll try using some old fashioned document generators to see if there's some sort of oddity, like defining a function at the bottom of a procedure file that is also a standalone PRG, or a class name that's defined in two places. There's 40 years of code in there, but I only noticed it missing files a few months ago, so I'll focus on modules with newer date stamps and work backwards.

I'll report any findings when I work it out in case anyone runs into a similar problem.


RE: Project Compiler suddenly stopped including newer files

Perhaps mark some suspects "Exclude"??

I know you have many many files, but perhaps they can be marked so in the pjx file. Don't know, just a guess.

Steve

RE: Project Compiler suddenly stopped including newer files

Quote (Joe Crescenzi)

So something in the middle must be stopping it from adding anything past a certain point in main.
I modified my test project so that programN calls ProgramN+1000, if you know what I mean, to check whether I see depth first analyzing of the code or breadth first, and the records in the pjx are still main, then program0001 to program2000, not programm0001, program1001, program0002,... so I don't see that.

You said...

Quote (Joe Crescenzi)

I used to just run a quick PRG I called "MakeVovio" to copy everything to a staging folder, generate the EXE from the project
Are you waiting for all copies to finish and be complete files in the staging folder before you start the build with a BUILD commad?

Depending on what you use for copying code can finish and the OS/Filesystem has still work to do.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


I know you have many many files, but perhaps they can be marked so in the pjx file. Don't know, just a guess.
Steve
Technically, since I'm doing my testing on a copy of my original, I can just intentionally delete the files with the newest timestamps, and then see if there's any improvements to narrow down the specific date that may have an impact. Once I see improvements, copy some back until it breaks the compiler and then I should be able to find the one(s) that could be the cause.

Quote:


I modified my test project so that programN calls ProgramN+1000, if you know what I mean, to check whether I see depth first analyzing of the code or breadth first, and the records in the pjx are still main, then program0001 to program2000, not programm0001, program1001, program0002,... so I don't see that.
Interesting. It's still hard to know for sure what role sequence plays, but I can absolutely say that when I look at the live PJX, the newest programs are definitely at the bottom and that when I start with an empty project, the order I see is definitely starting with the first objects found in main, and that that there were only 4 errors that are part of Main.prg, and all 4 are the last 4 DOs.

It's a real mystery and since there's a predictable pattern, where it always builds a project of 252, instead of 1400+, I'm hoping I only need to check those 252 files to find the answer. Like I said, it could also be the file that would've been #253. To test for this, I can write a program to copy the 252 to another test folder, then add another DO at the end and see if it finds it.

Quote:


Are you waiting for all copies to finish and be complete files in the staging folder before you start the build with a BUILD commad?

Depending on what you use for copying code can finish and the OS/Filesystem has still work to do.
Technically, I do it in two parts. First I run a batch to do the copying, then the MakeVovio is a PRG within the folder that builds the program and generates my Zip files to deploy updates.

I do this in two parts because I like to keep a copy of the last version deployed, including the source frozen in case I need to confirm or test any issues found in the last update. Otherwise I can't re-create or test problems 100% if I test it using source that is newer.

So, I always build the shipped version on a copy. I tend to keep the last 2-3 updates until I no longer need them, and it also lets me revert to a known safe version of a change in case something goes wrong with an edit. I also keep up to 31 copies of my code on a NAS that makes it easier to compare changes over the past month (I use a great compare tool for this) for the same reason.

The MakeVovio fails even in the original location, and all the copies of the directory I've made for testing this. In this case, all my recent tests are either in a temp folder on C:, or the exFAT drive I created to eliminate the potential for permissions.

I've done all subsequent tests on copies made earlier in the week. All versions yield 252 rows when using a fresh project. I've deployed perfectly working updates to 3 clients this week using the project file that I've been manually adding any files that are reported undefined. Thankfully, that works, but it still bugs me knowing that the compiler will not build fresh project files. I'll keep tinkering and see what I find.

RE: Project Compiler suddenly stopped including newer files

Quote (Joe Crescenzi)

when I look at the live PJX, the newest programs are definitely at the bottom
Yes, new records are last.

Quote (Joe Crescenzi)

when I start with an empty project, the order I see is definitely starting with the first objects found in main, and that that there were only 4 errors that are part of Main.prg, and all 4 are the last 4 DOs.

Obviously that's more interesting. I was also thinking does VFP maybe PACK the PJX data with a certain index order set to order by filename. Also since, the project manager lists prgs alphabetically, no matter in which order they were created/added to the project.

If you open up the debugger and its debugout window during compilation you can see it's analyzing full files, so that speaks against depth first processing. My build text project - even in the more compicated structure - will not do depth first analysis, so all programs are added merely because main.prg calls them all. That all likely differs in a project you maintain and grow over 40+ years, of course, but when you copy over to a staging folder the way the build process works and adds in files of course matters and it's good to know. Since I can't confirm what you see in that respect.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
It makes sense that the order in an older project file will seem random at first, then chronological over time because VFP definitely just adds things to the bottom when it encounters something new. As you pointed out, they can sort it on the fly anyway.

So as it stands now, the order that it parses main is my only lead.

I only know for sure that the top rows are definitely exactly in the order of the first DOs, and that all 4 unknown records just happen to be the last 4 DOs in main, so I can confirm that it doesn't just add all the DOs in main first before crawling them.

I could really shake things up by moving those last 4 DOs to the top, then seeing if it includes them this time, and leaves out different DOs at the bottom that were originally included, or removing the DO just before the last 4... perhaps the DO that is currently 5th from the bottom is the cause. So many fun experiments left. :)

RE: Project Compiler suddenly stopped including newer files

The depth/width first approach of parsing code can be tested with a much smaller project, and I did this:

1. main prg:

CODE --> main.prg

dothings()

Procedure dothings
   Do program1
   Do program3
   Do program4
   Do program5 
2. program1.prg:

CODE --> program1.prg

Do program2 
program3 and 5 are just RETURN and I didn't write program4, so a real missing program is involved now. To see whewther that stops analyzing mein.prg and not including program5.prg

Only main.prg is part of the PJX before building, then debugoutput says:
Build Project: Program main
   Module:  dothings
Analyzing: MAIN
   Module: DOTHINGS
Analyzing: PROGRAM1
Analyzing: PROGRAM3
Analyzing: PROGRAM5
Analyzing: PROGRAM2
Build Application: Program main
Build Application: Program program1
Build Application: Program program3
Build Application: Program program5
Build Application: Program program2 

And, finally, using the pjx file as table I see the files added after main are program1,3,5,2 in that order, just as the debugout lists it. If depth first would apply program1 should be analyzed when its DO call is found and since it calls program2, program2 should be added to the project a second, but it only comes in last, because first all programs called in main.prg are added to the pjx, only after that's done program1 is analyzed and program2 is found to be called and therefore included.

A single experiment can never prove any assumptions about, but I can reproduce this and haven't found an instance where depth first parsing/processing happened.

Also, program4 not existing didn't make the build stop, program5 is still included into the project, even before program2, which comes in last as expected for breadth first processing, or simpler said analyzing one program after the other. It's also telling me there's no pack and sorting done in the pjx records, that's only done in the project manaager listing of programs.

Quote (Joe Crescenzi)

None of the DO calls after a certain point were found, meaning it stopped looking at some point before reaching the end, which also means it doesn't just add all the direct referenced files before parsing subsequent files.
Well, I don't see that happening. There must be another reason why only 252 of about 1400 files are added.

That shouldn't stop you from trying what you do, but it may speak for a totally different reason stopping to include files and I hope you find it. Maybe you find something interesting in the debugout window, when you open it duringa a build.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
I can see in your example that it did add everything in main before diving into the called program2 was added.

There is one more wildcard in play in mine, the main.prg has several declared functions and classes defined at the bottom that are frequently used. The four missing DOs out of dozens that at the bottom of the last function in the PRG.

So a simplified view looks like this:

CODE

**
** Main.prg

**
do while .t.
   ** setup my environment with variables that needs to be global
   userid = ""
   username = ""
   do asklogin
   do form mymenu  && returns a variable called CHOICE, based on what they select from the menu 
   do case
      case Choice = "ADD-CUSTOMER"
          do addcustomer
      case choice = "BROWSE-SOMETHNIG"
          do browsesomething
      case choice = "EXIT"
          do signoff 
   endcase
enddo

** I'll define some classes like this so I can pass and return a city, state, county, and zip as a single object
** 
define class CSCZ as custom
   City = ""
   state = ""
   County = ""
   Zip = ""

   ** The class may have some functions too
   **
   function xyz
   parameters something
   do case
      case something="1"
         do something1
      case something="2"
         do something2
      case something="3"
         do something3
   endcase
   return something
enddefine

**
**
function SomethingINeverActuallyCallDirectly
   do prg1
   do prg2
   do prg3
   ** etc
   ** etc
   do prg21
   do prg22
   do prg23
   do prg24
   do prg25
endfunction 

So the PJX will have all the DO and functions called from the main function (in main.prg), plus all the functions declared in the classes, and all of the DO Prg# calls except the last four... 22,23,24,25.

So, the last four are still a mystery.

In case you are curious why I have a function that is never called at the bottom of main that has nothing but 20+ DO calls, it's just a trick I've used for decades to force the compiler to bring in a bunch of stuff that is called by things like customizable reports (that are intentionally excluded from the project), I have have a DO FORM Artwork that has nothing but pictures on it, so the art gets embedded in the EXE for things like reports.

For example, I'll put a function like ShowPickupDays(customerID) in the box on a report. Since ShowPickupDays is only called by the FRX report file, the compiler doesn't know it exists. By putting all those report functions in that last function ensures they are included. It's worked that way for decades.

RE: Project Compiler suddenly stopped including newer files

Okay, but then what you experience is not what I experience about missing PRGs 22,23,24,25. In my experience main.prg is fully analyzed and all DO calls cause included prg files. And all that happens before any of these called programs (or scx/vcx, whatever) are analyzed themselves.

Your theory of something within the other files stopping the extraction of files to include into the project does ont happen in my experiment. I could try if it makes a difference to put some calls into a function, also if it's never called. I don't think VFP is that picky or clever, wheatever reason you assume behind picking the files it actually includes, because in my experience VFP even analyzes and compiles excluded files, the only thing the exclusoin controls is whether that file is embedded into the final app/exe or not. So there is nothing clever excluding files because code analysis shows they are in code calling them which is never called, there's no such analysis taking place.

I added a function in my main.prg sample:

CODE

dothings()

Procedure dothings
   Do program1
   Do program4
    
Function nevercalled()
   Do program3
   Do program5 
Program 3 and 5 are still included. And I doubt a compile error in program3 would effect program5 to be included or not, as you can clearly see from the include order main.prg is fully processed before program3, so program5 is already in the pjx before program3 is compiled and could cause any problem.

It would be good to find a way program5 would not be included, so that you could find a reproducible reason. Your theory it has to do with order of analysis doesn't fit the experiments, so I strongly doubt it's that. There's always a chance that some more specific setup or configuration option influences it, but the pure fact that not all files claled from main.prg are included into the project points out a source of a problem outside of VFP, because all my experiments don't show a problem of suddenly stopping to include further files at some point. I even tried to expülicitly stop the bukild process through an actually not existing file, and even that didn't stop it.

I bellieve you get to such an incomplete project state, but the reason behind it seems to be outside of VFP itself.

Chriss

RE: Project Compiler suddenly stopped including newer files

I don't see you mentioning VFP verion. I know VFP9 can handle unlimited lines in a prg, but there was a limit in earlier versions. I don't reacp if it was 65000 lines or that only was a limit even VFP9 still documents for number of variables and arrays (not array elements, though). I think there was a 64kb limit for fxp files or prg files, you're not by chance hitting that limit?

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
I'm definitely using the last VFP9 SP (9.00.0000.7423).

My latest experiment was pretty amazing in both expected and unexpected ways.

Since the main program only skipped four PRGs, I decided to just move those four DOs about 10 positions higher in the PRG to see if it would include them, and still skip whatever four are at the new bottom.

The result was simply nuts. The error file now has 72 undefined objects (in the main.prg), and the PJX dropped from 252 to 184 rows, a difference of 67.

The plot thickens. :)

Then I removed the 4 prgs... and it generated a PJX with 1374 rows... so apparently one of those prgs freaks out the build, and where it's located in the main determines just how deep it will crawl subbed files.

All four files compile without errors, so its not the code itself, but something not obvious.

Chances are it's just one of those four that confuses the compiler, so I'll just need to isolate one at a time to see which one is to blame. I'm super curious which one, and what in particular confuses the compiler so much that it decided to stop parsing the rest of the files without even a single error in the debuginfo window (there's nothing unusual there).

Once I know, I'll share the results.

What a ride.

RE: Project Compiler suddenly stopped including newer files

Quote (Joe Crescenzi)

apparently one of those prgs freaks out the build

I agree, of course. I can't for good or bad think about what could achieve that. Because even drastic code like CLEAR ALL is just compiled, not executed during a build. The only thing I can think of stopping a build is when the build encounters a first not found file and pops up the Locate file dialog that asks you to Locate, Ignore, Ignore All and Cancel. But since you posted lengthy .err report files, you do use the "Ignore All" option.

One more straw to grasp: Do you make use of the IDE setting "save with end-of-file marker"?

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
It actually isn't doing an ignore all. I like to see each error as it comes up rather than looking at the .err.

The .err for the 252 row projects is actually pretty short because it doesn't see about 1200 files.

I'll check into the EOF markers. I haven't had to deal with those in decades because as you remember FoxBase DOS used to have a bunch of Nulls at the end that I eventually removed as I'd come across them, but as I mentioned, perhaps an update to some of those third party editors and search / compare tools may have inserted something that VFP doesn't like in at least one file.

The strange thing is all 4 compile without errors, so it must potentially only exhibit this when crawling objects to build the project.

I'll be tied up today, but this is definitely something that should finally be solved today.

RE: Project Compiler suddenly stopped including newer files

Quote (Joe Crescenzi)

It actually isn't doing an ignore all.
It? How do you then go through hundreds of errors, even more than 1000 missing files?
Do you CANCEL?

Regarding EOF marker: Helkp says it introduces CTRL+Z at the end of files and even if you turn this on I don't find that at all in the files, so I think you can forget that.

Chriss

RE: Project Compiler suddenly stopped including newer files

Hmm,

Joe , do you use project hook or not?

mJindrova

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


It? How do you then go through hundreds of errors, even more than 1000 missing files?
I meant I. I normally don't click ignore all, so I can write down errors as they come along. Normally when I build from a live project, it doesn't find anything except for some reason it has trouble understanding two of my array names that it thinks are functions, even though I always explicitly declare them as external arrays just before using them.

In this case, it doesn't actually stop over 1000 times. Remember, when I build from an empty project, it only see the first 252 programs, so at that point the compiler has no idea those other 1200 or so files exists, so it never parses or looks for most of them because they're called by just the ones that were found.

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


Joe , do you use project hook or not?
No. Never tried it, but it would be funny after nearly solving this to discover there was a tool to help me find the answer.

RE: Project Compiler suddenly stopped including newer files

Quote (Joe Crescenzi )

I meant I. I normally don't click ignore all, so I can write down errors as they come along.

Well, for once: After the builkd VFP will pop up the .err file, so there's no reason to note down one error after the other.
If you cancel, are you not awre that you are cancelling the build and thus cause the rest of the PRGs to not be processed and n wonder you don't get a complete project at all.

I mean, you already showed longer lists of undefined functions and not found files, they have to come from builds you did with ignore all, don't they?

I'd always recommend to ignore all unless you have to do with single files not found you can locate, but even that is better fixed in the project than during the build process. See how far you get and what's really finally missing when you build ignoring all errors. It doesn't mean they will not be reported, you get a full overview of problems if you build that way. And, of course, you get an incomplete project if you stop.

Chriss

RE: Project Compiler suddenly stopped including newer files

Hello,

strange.
Maybe you can watch the compiler run with process explorer which reports any fileactivity, maybe an access denied by an AV program,...
procmon (freeware, portable) from sysinternals/MS : https://live.sysinternals.com/

regards
tom

RE: Project Compiler suddenly stopped including newer files

Quote (Joe Crescenzi)

Remember, when I build from an empty project, it only see the first 252 programs, so at that point the compiler has no idea those other 1200 or so files exists

Don't you see the error in your line of thinking? When you stop build, of course VFP doesn't analyze all the further PRGs and doesn't incude all further files, not only the ones after a certain line in main.prg, every other file. You only get a complete overview of what's found and not found when you complete a build run by using the ignore all option when the first file/function/class/whatever is not found.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


Well, for once: After the builkd VFP will pop up the .err file, so there's no reason to note down one error after the other.
If you cancel, are you not awre that you are cancelling the build and thus cause the rest of the PRGs to not be processed and n wonder you don't get a complete project at all.

I mean, you already showed longer lists of undefined functions and not found files, they have to come from builds you did with ignore all, don't they?

I'd always recommend to ignore all unless you have to do with single files not found you can locate, but even that is better fixed in the project than during the build process. See how far you get and what's really finally missing when you build ignoring all errors. It doesn't mean they will not be reported, you get a full overview of problems if you build that way. And, of course, you get an incomplete project if you stop.

Chriss
In most cases with my live project, I get exactly two errors, just the two arrays that for some reason pop up in spite of declaring them as external arrays in the line just before they're being called, so I don't mind clicking ignore twice.

I never cancel.

Since I've been tinkering with the sandbox copy of my code, I always got the same 252 rows whether I ignored all or one by one. I tried it both ways, so my interaction was never a factor. What did turn out to matter was moving or removing the four last DO calls.

All four compile with zero errors, so it's going to be interesting to see why one of them chokes the compiler. As I mentioned, my code is tight and I'm the only developer and the only third party code that lives in this project is XFRX, but as I mentioned, I deleted it in the test folder with no change, so it's not an overlapping function or object name confusing the compiler.

So here's where things stand.

With those four files as the last four DOs, it always builds a project with only 252 records.
With those four files moved to an earlier position, it skips about 70 more files, and the project shrinks to about 180 records.
With those four files removed, it builds a project with nearly all 1400 files.
One of the four is just XFRX, so I can eliminate that since I tried it with it removed with the project being reduced by just one, itself.

The remaining three are super short programs that literally do virtually nothing important.

Proc./Func. __ID2ID2 - Undefined
Proc./Func. CHANGEACCOUNTID - Undefined
Proc./Func. FIXORPHANS - Undefined

__ID2ID@ just copies a string (ID) to a second variable (ID2) if it's left out (there's a search box with a start and end. If they only enter a start value without an END value, it makes them the same). It's probably only 10 lines of code, at most.

CHANGEACCOUNTID just takes two parameters to change an account ID from one value to another. It checks to be sure the account ID isn't in use, and changes all the child tables accordingly.

FIXORPHANS just makes sure that there are no records in secondary tables that have no parent client ID. Theoretically, it never needs to be called because the ChangeAccountID has every conceivable child update fully updated. I only include it in the system with a backdoor hidden menu option in case some of the clients who've been using the system 25+ years ago still have orphans before I fine tuned the changeaccountID code.

So as it stands one of those is the culprit and none look like anything unusual. I'll explore them tonight and I'd really love to know the answer.


RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


Don't you see the error in your line of thinking? When you stop build, of course VFP doesn't analyze all the further PRGs and doesn't incude all further files, not only the ones after a certain line in main.prg, every other file. You only get a complete overview of what's found and not found when you complete a build run by using the ignore all option when the first file/function/class/whatever is not found.
I don't think you understood. I never stop the build. I always use Ignore and in all those sandbox starts, I did it once or twice with ignore one by one to see what it would do, then dozens of other times, I used Ignore All because I always got the 252 rows regardless of whether I hit ignore a few dozen times, or hit ignore all.

I always let it complete and it always generates an EXE (but with missing functions).

By removing the four DOs entirely, I do the same ignore or ignore all, but this time I got nearly all 1400.

The only difference is still the four files, and since one is XFRX, and I think I've already tried a version with JUST that removed from the main. To rule it out entirely, I also removed the XFRX.* from the sandbox, leaving just the remaining 3... all of which compile with zero errors when compiled individually (compile changeaccountid for example).

RE: Project Compiler suddenly stopped including newer files

Quote (Joe Crescenzi)

all of which compile with zero errors when compiled individually
The major topic is not finding/includng files, not errors in code, isn't it?

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
Exactly. None of the three have errors in the code. If I typed compile *.prg, there will be zero .err files, because none of the programs has compile errors.

The mystery is why one of those 3 programs is causing the compile to not include the majority of code.

Before all these tests, I used my real project file, and it was only missing the programs I created recently because the rest of the files were already in the project. When I add them by hand, I get a perfect EXE without any missing functions.

In summary:

By creating a new project with just the main, it always included just 252 files, and didn't parse the rest.

By moving the last four DOs, the project is even smaller, skipping about 70 more.

By removing the four DOs, I get virtually everything, 1377 rows in the project.

That's where I stand. I know those 3 files are super short and simple PRGs with nothing obvious in them. I'm sure it's just one that is the culprit, so I'll work out why and report it tonight.

RE: Project Compiler suddenly stopped including newer files

Okay, I'm stumped.

But going to mJindrova asking whether a project hook would cause this and your answer to that:

Quote (Joe Crescenzi)

funny after nearly solving this to discover there was a tool to help me find the answer.

I'd be curious if you write a project hooks AfterBuild event maybe just MESSAGEBOX("afterbuild") whether you get there or something actually interrrupts the build.

Then also the procmon tom mentioned started before build could list all files the vfp9.exe looks for and whether it finds them or fails. Both things could tell more about what happens during the build and whether you actually get to after the build.

Not to forget AV might intervene, especially since you build an EXE.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


Not to forget AV might intervene, especially since you build an EXE.
I only use MS Defender, but anything is possible. It's still crazy to think one of those 3 innocent looking PRGs is the root cause when they do virtually nothing and compile flawlessly on their own.

But, facts are facts. Moving the 3 DOs 10 lines higher makes it even worse skipping 70 more, and just commenting out those 3 DOs yields an almost complete project of nearly 1400 records.

RE: Project Compiler suddenly stopped including newer files

Hi Joe,

You say the 3 prg's are short. I know this makes no logical sense, but maybe recreate/retype them to replace the originals.

I suggest this because on rare occasions, (maybe 2 in several years) I have had to retype a perfectly good statement to avoid a compilation error.

Just a wild guess.

Steve

RE: Project Compiler suddenly stopped including newer files


Joe, how much branches has DO CASE... ENDCASE in main prg?

mJindrova

RE: Project Compiler suddenly stopped including newer files

(OP)
********* I FOUND THE PROBLEM
*********
********* DRUMROLL
*********
**
** It was the way XFRX was (accidentally) used.
**
*********

Many years ago, before I used XFRX to create PDFs using a ReportListener, I called a function that was part of an older XFRX library that didn't create PDFs from reports, but let you basically place text on the page programatically. It was close to what I did in the DOS days, which was to write PostScript code, which turns out to be pretty easy once you get the basics. You give it a string followed by coordinates starting from the lower left as 0,0 and told it to show it there. I loved the control I had, but when the ReportListener was developed and XFRX used it, I stopped using the old way.

To use it, you referenced it with "SET PROCEDURE TO XFRX" so it could find the classes whenever you needed to call those functions. Apparently, one of those remaining prgs still had a SET PROCEDURE to XFRX in the code.

To fix it, I just commented out that one line and I can now build a perfect project with everything.

I don't even remember how long ago I used XFRX as a procedure file, but there it was, and for some reason it freaks out the compiler if it's anywhere in the code.

I confirmed this multiple times in my sandbox. When I put that one line in the code it gives me the tiny PJX, when I remove it I get all 1400.

I thought for sure I eliminated XFRX from my testing because I did at least one test with the files removed, but I guess I missed it after all.

It was fun exploring all the possible options... thank you for taking the time to help me figure out this puzzle.

In the end, there's still one tiny mystery... Why would the compiler freak out when there's a reference to a PRG that otherwise works when you call it the regular way? If anything it should've added it, and the runtime would've barked over that unnecessary reference.


RE: Project Compiler suddenly stopped including newer files

Quote (Joe Crescenzi ()

Why would the compiler freak out
Indeed, as I already said any code that's compiled isn't executed at the same time, so how could it trip up the build process at all?

You could do one more experiment. Uncomment the SET PROCEDURE, but before you build issue a CLEAR ALL.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


Indeed, as I already said any code that's compiled isn't executed at the same time, so how could it trip up the build process at all?
I have one theory, but it's a stretch.

As I mentioned, I licensed it with source, which means the folder has XFRX.PRG and XFRX.APP.

It could be that the SET PROCEDURE TO XFRX triggered the compiler to crawl the PRG, even though there's already an APP in the folder and it became sort of a Clark Kent / Superman thing where the two should not exist in the same place. When it saw the same internal objects in both places it bailed out. It could even have something to do with the recent update I did, so the APP might be newer (or older) than the source.

In either case, I'll keep them apart and only compile a new APP if I decide to alter the code (Which isn't likely, or I'll need to re-mod every time Martin updates it).

Above all I made sure there are no longer references to the pre ReportListener way of declaring XFRX a procedure file.

RE: Project Compiler suddenly stopped including newer files

Joe,

Please, delete xfrx.app - it's a demo version.

https://eqeuscom.atlassian.net/wiki/spaces/DOC/pag...

2.1. I am evaluating XFRX. It works fine in development, but when I try to compile the demo in my project, the VFP quits or crashes. Is that because it is a demo version?
This is an issue in the demo version of XFRX - it cannot be included into VFP projects, it makes VFP crash (this is a negative side effect of a decompilation protection). The workaround is to invoke XFRX via macro substitution:
loSession = EVALUATE[XFRX('XFRX#INIT')]
This way XFRX.APP does not get into the project and you will be able to compile without problems. The commercial version is not affected by this problem.

mJindrova

RE: Project Compiler suddenly stopped including newer files

Thanks, mJindrova.

That explains a lot.

Aside of that, Joe, why would you let an app file be part of a project? I don't see any case where an APP would be included in a build and work embedded into a final EXE. Take the usual report engine, not even foxypreviewer, you're asked to deploy the app files separate to your EXE as part of the runtimes. So even if an app would not cause build problems, it should always be kept separate. The only reason to include an app in the project itself, but excluded, is that it doesn't cause unresolved references.

That adds to the build from scratch only having the main.prg in the project before building: You either also add in the app file and set it to excluded before the first build or you keep out any references to the APP file from any code, so it's not included in the build. No matter if there are full sources included or not.

Chriss

RE: Project Compiler suddenly stopped including newer files

Joe,

Glad you found the problem.

I never include an app file in a built, it’s allways distributed separately.

Regards, Gerrit

RE: Project Compiler suddenly stopped including newer files

(OP)
I've never included the App implicitly in my Project because I've always included the App externally in my distribution.

The project worked perfectly until I somehow brought in an older PRG that had a "SET PROCEDURE TO XFRX" line in it.

The project file has worked for decades, but one of the 3 last DOs in my main program had that line because the version of XFRX created before VFP introduced the ReportListener class was done via a "Set procedure to XFRX".

For what it's worth, once I removed that one line, I can build my project without it skipping anything.

In summary, with the Set Procedure To XFRX line there, it breaks the compiler. The earlier that line is in the program, the more files it would skip. Removing it works perfectly.

Meanwhile, I recently downloaded an even newer version of XFRX, so I'll be very careful to remove the old one and I know for sure I don't have any other references to XFRX using the older SET PROCEDURE method.

RE: Project Compiler suddenly stopped including newer files

Joe,

a SET PROCEDURE TO something does not only address something.prg, it also adddresses something.app and includes an APP file if one is found. You can check it out with a new project that has a main.prg which does nothing else but

CODE

set procedure to reportoutput 
.
You'll see that building that will include reportoutput.app from HOME() into the project, excluded is automatically set.

The rest then is obvious with mJindrova's explanation.
And since excluded is the default I think the build problem also occurs in that case. Indeed when you open the debugger before build and open the debugout window, you can see VFP processes excluded files, too. It's not a propblem with any app, so you have to know it's caused by the copyright protection of XFRX.

Just for sake of demonstrating what happens if an app is included in the build process: Create a new empty project, create a new main.prg with just SET PROCEDURE TO REPORTOUTPUT.
open up the debugger with Tools->Debugger and open the debugoutput window.
Build the project and debugoutput should output this:

CODE -->

Build Project: Program main
Analyzing: MAIN
Analyzing: FRXOUTPUT
   Module: ReportOutputCleanup
   Module: TestListenerReference
   Module: GetSupportedListenerInfo
   Module: ReportOutputConfig
   Module: GetConfigObject
   Module: ReportOutputDeclareReference
   Module: UnloadListener
   Module: HandleError
   Module: CheckPublicListenerCollection
   Module: Execute
Build Application: Program main
Build Application: Application reportoutput 

You see:
1. even though reportoutput.app is included into the project as excluded project item, it is analyzed.
2. All the lines after Analyzing: MAIN are about reportoutput.app and you can see VFP goes into modules of the reportoutput.app.
All that's also happening with XFRX.app once that's included into the project. And I guess one of the last lines in the debugout of your failing build would have been about a module of XFRX.app.

Notice how hard that is to detect even in this well known case: The line after analyzing main is not stating to analyze reportoutput.app or reportoutput, it says:

CODE -->

Analyzing: FRXOUTPUT 
So it may not become clear the build hangs on XFRX.app

It becomes clearer once you look into the reportoutput.pjx coming with XSOURCE.ZIP. In that project the main program is called FRXOUTPUT.PRG, so that's what's analyzed from an APP file, the embedded main file of the app. It's not easy to spot and you need to know the sources of an app to realize that. That needs to be known on top of knowing that SET PROCEDURE TO xyz also addresses xyz.app, which is not mentioned in the help topic of SET PROCEDURE. So all in all that's it, but even if I knew what mJindrova knows about the source code protection of XFRX. If the XFRX.app main file also is main.prg it becomes even worse, the debugoutput would state to analyze main once more and you never would suspect that means main.prg inside XFRX.app, but who knows, I don't have XFRX to check it out. The cherry on top is that you stated XFRX can't be the problem since you excluded usage of it. Well, obviously you didn't remove it completely.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


The cherry on top is that you stated XFRX can't be the problem since you excluded usage of it. Well, obviously you didn't remove it completely.
I did so many experiments and I thought I tried that at least once, but several people mentioned that XFRX wasn't likely the cause because it's been well tested over the years. I just didn't realize one of my PRGs had that set procedure to xfrx in it, or that it would do anything other than reference the same files it already referenced.

For what it's worth, I'm going to do a deep dive as to the latest XFRX docs to see what's changed over the years. I've used it for many years without problems, but there could be new best practices that I'm not aware of in the docs because I haven't needed to read them in a very long time. Who knows, maybe the latest version has new things I should explore.

I'm also going to put a preferred report viewer setting into my system so I can swap between multiple report listeners to see which ones work best, or even let my customers choose which ones they like on the fly.

Right now all my reports are passed through a single function I simply call RepView.prg, so I should be able to do a simple do case without having to modify hundreds of potential places where reports are called. Then I can easily try other report viewers and pdf generators like the newer FoxyPreviewer, which I couldn't do previously because I have a high DPI screen (the older version would mess up the page size if you have a high DPI monitor).

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


mJindrova (Programmer)18 Mar 24 19:58
Joe,
Please, delete xfrx.app - it's a demo version.

2.1. I am evaluating XFRX. It works fine in development, but when I try to compile the demo in my project, the VFP quits or crashes. Is that because it is a demo version?
This is an issue in the demo version of XFRX - it cannot be included into VFP projects, it makes VFP crash (this is a negative side effect of a decompilation protection). The workaround is to invoke XFRX via macro substitution:
loSession = EVALUATE[XFRX('XFRX#INIT')]
This way XFRX.APP does not get into the project and you will be able to compile without problems. The commercial version is not affected by this problem.

You are correct, the demo version as distributed as an APP. The live version is distributed as FXP by default, but if you pay extra for the source, it's a PRG. I use the source version, so I have a PRG, but I compiled mine to an APP, so my APP is a live version, although technically, if I delete the APP the compiler will add the PRG into my project and it should work the same, but make the exe a bit bigger. It's already 14MB, so it wouldn't be a big change.

Here's how I call it in my report viewer PRG.

CODE

loxfrx = XFRX("XFRX#LISTENER")
  loxfrx.targetType = "PDF"
  loxfrx.targetFileName = MyFileName
  lnRetval = loxfrx.SetParams()
  IF RECCOUNT() = 0
    VovioMessage("This report is empty.  Try changing the filters")
    RETURN ""
  ENDIF
  IF lnRetval = 0
    REPORT FORM (_RepFile) OBJECT loxfrx
    loxfrx.finalize()
    RETURN MyFileName
  ELSE
      ? "Invalid output folder/file", MyFileName, lnRetval
      WAIT WINDOW
  ENDIF 

RE: Project Compiler suddenly stopped including newer files

Joe,


if you compile xfrx to separtly app, then the same rules apply as for the demo.

mJindrova

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


if you compile xfrx to separtly app, then the same rules apply as for the demo.
mJindrova
Then that makes it even easier for me to decide what to do. I can just delete the APP like you said and just call the PRG.

For what it's worth, if that's the case, it's always been pulling the PRG into my project and the runtime most likely ignores the APP.

The other thing I've learned is that the SET PROCEDURE TO XFRX is to be avoided at all costs.

RE: Project Compiler suddenly stopped including newer files

I don't use XFRX, but now I'm interested, too.

Quote (mJindrova)

if you compile xfrx to separtly app, then the same rules apply as for the demo.
But you also said:

Quote (mJindrova)

...XFRX.APP does not get into the project and you will be able to compile without problems
...
The commercial version is not affected by this problem.

So I'm still not getting this straight: Does or doesn't the commercial XFRX.app make VFP crash when building a project?

Also, from my side, Joe:
Since SET PROCEDURE still prioritizes PRGs over APPs, it does not necessarily cause the XFRX.APP to become included into your project. That must have been something caused by files available and not available and seen by the build process. So, in the end, if only the demo app causes build crashes, you could still have SET PROCEDURE TO XFRX to either cause including XFRX.prg or XFRX.app or both, not sure how that would turn out. Because even if a SET PROCEDURE TO XFRX.prg would explicitly only include the prg file, if that in turn contains code that uses the APP without the macro substitution workaround, it would include the APP anyway.

Now, it only depends on your clarification, mJindrova. Is this really only a demo version problem or to be avoided generally?
And on top of that, would even direct usage of a prg, not the app, include files eventually causing the build problem, so you don't just have to avoid SET PROCEDURE TO but any direct usage? I can't imagine tha latter to be "user" (=programmer) friendly.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


Now, it only depends on your clarification, mJindrova. Is this really only a demo version problem or to be avoided generally?
Chriss
I was wondering the same thing. I was planning on making another APP like I did in my older version, but now I wonder whether that file was being ignored (because I also had xfrx.prg in the same folder)... or used. If using it as an APP makes it a demo, then I have no choice.

Although I've renewed my source code version multiple times over the years (mainly to support an independent developer), I've apparently been using a 2010 build because I never quite got the source to work on newer versions and I completely forgot which one was actually being called.

So, in the build that has been working for me (using an APP I compiled in 2010), there are no errors as long as I don't use the dreaded set procedure to xfrx.

Meanwhile, as I explore my working folder, I decided to finally upgrade to the latest version (Feb 2024)... and wouldn't you know it... the project file is a train wreck again.

The program runs flawlessly after adding about 30 files, a mixture of files like DLLs, FLLs, VCX, etc.

Compiling it to a fresh Project yields a bunch of things, starting with this:

CODE

Program c:\sourcecode\cartpro.new\xfrx.prg has the following errors:
    Unknown _XFGETLTI - Undefined
    Unknown _XFBMP2 - Undefined
    Unknown _XFCROPIMAGE - Undefined
    Proc./Func. XFRX_EXTRACTFILEFROMAPP - Undefined
    Proc./Func. XFRXPROXY - Undefined
    Program XFRXPROXY - Undefined
    Proc./Func. XFRX_TESTFILEINAPP - Undefined
    Proc./Func. XFRX_USETABLEFROMAPP - Undefined
    Proc./Func. XFRX_SETPICTINAPP - Undefined
    Unknown _XFCONVERTIMAGE - Undefined
    Unknown _XFIMAGEBPP - Undefined
    Unknown _XFWMF2IMAGE - Undefined
    Unknown _XFWID - Undefined
    Unknown _XFDEC - Undefined
    Proc./Func. XFRX_EXTRACTREPORTFROMAPP - Undefined
    Proc./Func. XFRX_RUNMACROINAPP - Undefined
    Proc./Func. XFRX_RUNPROGRAMINAPP - Undefined
    Unknown _XFIMAGES - Undefined
    Unknown _XFGETVERSION - Undefined
    Unknown _XFIMAGEGETDPI - Undefined 

Meanwhile, after that bunch of errors, it starts say about 150 or so of my PRGs are undefined again, so the project loses another 150 prgs again. It's not as bad as only having 252 records, but clearly not an easy update from the 2010 version that generated no undefined objects just prior to trying to update it.

So, if the solution is to generate an APP first, like I did in 2010, I really need to know whether that means that the code runs like a trial version when made into an APP or not, because I just want it to work as it did before.

RE: Project Compiler suddenly stopped including newer files

I think compiling an app from full source should not create an app that crashes VFP. In general, you can have apps in projects without getting build problems.

Then what still happens is when you call prgs from an app not included into your own project, you get undefined error messages, but can ignore them, as they are contained in the external app. To not get compile errors you'd define things in a #IF .F. #ENDIF section to keep the parser silent about undefined functions that are still present in the app.

Or you go the completely opposite direction and don't build an APP, instead include the XFRX sources into your own project so they become integral part of the EXE.

mJindrova,
I still look forward for clarfications and advice from you.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
It doesn't bother me that the compiler considers all those objects undefined.

What bothers me is that now the compiler is missing over 150 of my own files again, and will likely stop bringing in new stuff just like it did when this whole thing started.

So, unless I figure out how the newest XFRX needs to be integrated, I'll go back to the 10+ year old version and put my code back the way it was... except for that one line that broke the compiler in the original.

RE: Project Compiler suddenly stopped including newer files

Quote (Joe)

missing over 150 of my own files again
To me the list looks like only mentioning XFRX components. You know best what's missing from your own files, of course.

I wonder why you would even need to include anything from XFRX to call something in an APP. Or in the alternative case, when you do loxfrx = XFRX("XFRX#LISTENER") why you would build an APP at all. There should be documentation about how to use XFRX, shouldn't there, also from the point of view of what files to add to your own projects, if any.

Chriss

RE: Project Compiler suddenly stopped including newer files

https://eqeuscom.atlassian.net/wiki/spaces/DOC/pag...

Important note
The evaluation version of XFRX cannot be included into VFP projects, it makes VFP crash. To avoid this please invoke XFRX via macro substitution:
LOCAL m.loSession
m.loSession = EVALUATE("xfrx('XFRX#INIT')")
 
This way XFRX.APP does not get into the project and you will be able to compile your application without problems.

RESULT: If Joe compile xfrx.prg to xfrx.app then must initiate XFRX object follows: loxfrx = EVALUATE("XFRX('XFRX#LISTENER')")


mJindrova

RE: Project Compiler suddenly stopped including newer files

(OP)
I only pasted the XFRX missing parts, which in itself is a sign that you really can't just use the XFRX.prg in your project without a struggle. The rest are about 100 or so PRGs which are in plain sight in the same directory, but since the compiler breaks... are never found, nor are their nested calls. This time it's not as many, but it's still quite a bit to add back by hand.

From what I can see, I still have a project file I made in 2013 to build the (easier to work with) version from some time around then.

The instructions for installing and using XFRX are different from distributing it, which adds to the confusion.

The distribution instructions contain a list of this is dependent on that, which in turn is dependent on something else. Some of the files are in (home)\ffc, others are inside of some zip files along with other files, and others are simply various GDIPlus, and zip libraries.

What I'd really like to see is a folder called "COPY-TO-YOUR-DEV-FOLDER-TO-COMPILE" and/or "COPY-TO-YOUR-DISTRIBUTION". Frankly, I'd love to see it coexist with my own code, but considering all the problems, I'd love just a simple handful of pre-compile files that I can just drop into my dev folder and go. I located about 30 required files and it runs, but as I said, that seems to break the compiler again and it stops looking at my own code files.

I'll also need to make sure that even if I manually add any missed code, that the distribution will work anywhere. For example, he instructs you to add a "SET CLASSLIB TO (HOME()+"ffc\_reportlistener")" just before calling it. I need to ensure that the compiler pulls that into the compiled exe because my clients won't have a VFP IDE with an FFC folder (I don't think the runtime includes it).

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


RESULT: If Joe compile xfrx.prg to xfrx.app then must initiate XFRX object follows: loxfrx = EVALUATE("XFRX('XFRX#LISTENER')")
mJindrova

Thanks for clarifying this. So, when you said "The evaluation version of XFRX cannot be included into VFP projects, it makes VFP crash.", the bottom line is you can't include ANY XFRX.App, even a compiled version of the paid version.

So, basically any .APP version of XFRX will crash the compiler.

In the last version I used, I did not bring over an APP. I just added the XFRX.PRG and about 30 other files. That was a mess and it didn't compile correctly, so I'll try to put everything into one folder, compile it into an App, and try the macro substitution.

RE: Project Compiler suddenly stopped including newer files


Joe,


XFRX#LISTENER is derived from vfp report listener and depends on next files which are not parts of xfrx distribution package.

All writes here:
https://eqeuscom.atlassian.net/wiki/spaces/DOC/pag...

mJindrova

RE: Project Compiler suddenly stopped including newer files

Quote (Joe)

he instructs you to add a "SET CLASSLIB TO (HOME()+"ffc\_reportlistener")" just before calling it. I need to ensure that the compiler pulls that into the compiled exe because my clients won't have a VFP IDE with an FFC folder (I don't think the runtime includes it).

That shows some missing knowledge about how things work during IDE usage of code and within a final EXE. If you SET CLASSLIB To any directory not existing in the target computer (the end users machine) then that still just looks inside the EXE and the build includes the _reportlistener.vcx into your project.

Quote (Joe Crescenzi)

you really can't just use the XFRX.prg in your project without a struggle
You only have one main.prg in any pproject, so if you expect moving the main program of a separate project into your project would work the same way as starting with just your main.prg

Overall, it sounds like it could be better documented, but your way of building up a fresh project everytime you build also isn't the normal way to work in projects and it seems to me all your problems mainyl stem from that way of working.

Chriss

RE: Project Compiler suddenly stopped including newer files

Joe,

Being a XFRX user for more than a decade I allways try to use the latest version. First in a special test application I built for this purpose, then in our main applications.

Certain versions of XFRX have had their issues, as you can see in the XFRX release notes. The latest versions work perfect for us and our customers.

So you should be able to get this running.

Regards. Gerrit

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:


Overall, it sounds like it could be better documented, but your way of building up a fresh project everytime you build also isn't the normal way to work in projects and it seems to me all your problems mainyl stem from that way of working.
I'm not sure if I made this clear. I haven't needed to generate a new Project file in at least 15+ years. I've always use the same project file.

This whole thing started a few months ago, when an older prg made its way into my project that had "Set procedure to XFRX", causing XFRX to compile into a dead end, and then it stopped finding any new files. I kept adding anything new by hand, but wondered why it stopped automatically crawling my code to add anything new. That's when I posted here to see if anyone else had that happen.

So, in order to isolate the problem, I copied my development folder to a different drive, and tried to just build a project from scratch. Every time I did this, it picked up exactly 252 files, and didn't crawl and discover anything else.

With that in mind, I systematically kept tinkering until I realized that from the main itself, there were only 4 missed files, and all four were at the bottom. By moving them higher, it was even worse, skipping at least 80 more files... removing them made it find everything (about 1400). That's how I knew I only needed to explore those 4 files. Eventually I discovered the set procedure line and removed it and it worked as it did for decades, capable of finding everything.

With everything working, I was inspired to finally bring in a newer XFRX, but by using the PRG, the original problem is back. I'm going to take mJindrova's advice and compile it to an App and only reference it by macro substitution. The main thing is not allowing the compiler to parse XFRX.

RE: Project Compiler suddenly stopped including newer files

(OP)
Here's where things stand.

I changed all references of:

m.loSession = xfrx('XFRX#INIT')

To

m.loSession = EVALUATE("xfrx('XFRX#INIT')")

Now, the compiler doesn't even know XFRX.PRG exists.

Then I simply compiled XFRX.PRG into XFRX.APP. It generates a bunch of errors, but the APP works perfectly, and in my sandbox no longer messes up the compiler.

I'll just need to make sure I include the .APP and the other required files are in my distribution and that should do it. From this point, I should always be able to swap out the newest XFRX by pre-compiling to an APP like I did here and things should be good. :)

RE: Project Compiler suddenly stopped including newer files

Good to hear you found a solution.

Sorry, that I took it for granted you build this way, since you mentioned a statging folder and this method seemed to me motivated by only inclluding everything into a project that's actually used.

Good news, or bad news - depends on how you look at it - I just tried to start a simple project only doing SET CLASSLIB TO _reportlistener.vcx, not literally, I used both the syntax with HOME() and the absolute path and most of the time, even though the _reportlistener.vcx file clearly exists, the VFP project manager reported to not find the file and the vcx was not included into the project. So there is so,mething going on with VFP not adding files into a project, even when they are explicitly specified in source code.

That means the issue you have with too small projects could be an issue completely independent of the XFRX issue. As if there is a change in the file system api regarding the way VFP looks for files.

To reproduce the problem, all you need to do is start VFP fresh, default directory (for me) then is HOME(), i.e. ? Sys(5)+Sys(2003) prints the same as HOME().
Creating a new PJX with just SET CLASSLIB TO ("c:\full\absolute\path\to\_reportlistener.vcx"). Build and the vcx is not included, sometimes with reporting the "undefined", sometimes not.
Only when CD into the ffc folder before the build, the vcx is found. If that's normal, I must have added anything I used manually, so far. Of course it's never a problem. And if I think of how I usually introduce a new PRG, I write a new PRG by using the NEW button of the project manager, which means the new prg is added that way before build. Anyway, it's a well known feature of VFP to include files during build and the experiment with the 2000 programs also worked that way. There was no MS update since then. So whatever does not work is unclear and the exact circumstances, too. And, well, this is Windows 10, not even 11:

Edition Windows 10 Pro
Version 22H2
Installed on ‎11/‎03/‎2022
OS build 19045.4170
Experience Windows Feature Experience Pack 1000.19054.1000.0

I'm not sure, but it seems not normal, even if the absolute path is not within the project home directory. If you use vcxes, prgs from several folders within Home() or your own directories, you can't obvioulsy CD into all of them. What also works is Set Path To ".\ffc" additive, but that means PATHS relative to the default directory you set when building, not relative to the project home directory. That's not new, I already said build can depend on the current/default directory, but then absolute paths are absolute paths. I wonder what is going on.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
As a followup... Distribution.

Unfortunately, since "SET CLASSLIB TO (HOME()+"ffc\_reportlistener")" is evaluated at runtime, it didn't just pull it in, so my test distribution failed to find it when I ran the exe, so I copied it locally, along with a few other files referenced from the (external) XFRX.APP, including GDIPlus and added them to the batch that builds my distribution and everything is running smoothly.

The newer XFRX seems to work faster than the older version, and the PDFs somehow look cleaner. It's hard to say why, but the margins seem more accurate, formatting and some of the typography looks cleaner, and they handle some things the older one couldn't do, like angled text.

I just need to confirm important things like addresses showing up invoices in the same places, so they fit windowed envelopes the same way, etc.

RE: Project Compiler suddenly stopped including newer files

Quote (Joe Crescenzi)

so I copied it locally,...

I would rather include it manually into the project - especially since what I found is current behavior with automatic including project files, see above - it then will work without any external extra files and no matter if HOME() exists n a target machine or not.

Just double checked, once the _reportlistener.vcx is in the project (added manually, for example) you can even write

CODE

SET CLASSLIB TO "I don't care\_reportlistener" 

This will only cause an error when you run this PRG within the IDE, the EXE will ignore any path, even HOME() and just look for _reportlistener.vcx in the EXE, find it and not error.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
I'm not surprised (home()+"\ffc") didn't work. Because of the parenthesis, that's just another macro, so it's evaluated at runtime so the compiler didn't bring that over. I guess I could've used the physical path because most VFP installs map HOME() to the same place, but it was easier to just copy over.

My method of staging builds in different directories is just an audit trail and it helps me reproduce problems customers may find in earlier versions.

On a simplified level it goes like this:

I develop in a folder with .NEW that I never compile from.

When I want to distribute a new build, I like keeping everything from that build together frozen in time, so that goes to a staging folder, normally with a date like .319. I compile in this folder, and if there are no build errors, I run a batch file to zip up a distribution.

If a customer calls with a problem and I can't recreate it using the newest version, I like to re-create it running the same version of the source they have installed (3/19) to see exactly what they see. Also, since disk space is dirt cheap, by keeping the builds from the last 30 days, its also easier to compare changes. Technically, I also periodically push code to a GitHub repository, so I can compare unlimited older versions of a file that may have gone off the rails a long time ago, but sometimes it's quicker to just compare local folders. I use a great program called ExamDiffPro for that.

RE: Project Compiler suddenly stopped including newer files

Joe,

you miss that I said I used an absolute path and replaced HOME(), so that's not the fault.

There really is a bug about including files that are not in the default directory or SET PATH, even if they are specified with absolute path.

And I know from the past experience HOME() is a constant for VFP, so it is not only evaluated at runtime. It shouldn't be a problem, just like an absolute path shouldn't be a problem. There's something on top of the XFRX difficulties hindering VFP to include files into a project when it's not manually added. And the only chance that is normal is I never used this as I always created or added necessary files before building a project anyway.

Chriss

RE: Project Compiler suddenly stopped including newer files

Joe,


Yes, because vfp can't find procedureas and function from other parts:
A file xfrxproxy.prg
    Proc./Func. XFRX_EXTRACTFILEFROMAPP - Undefined
    Proc./Func. XFRXPROXY - Undefined
    Program XFRXPROXY - Undefined
    Proc./Func. XFRX_TESTFILEINAPP - Undefined
    Proc./Func. XFRX_USETABLEFROMAPP - Undefined
    Proc./Func. XFRX_SETPICTINAPP - Undefined
    Proc./Func. XFRX_EXTRACTREPORTFROMAPP - Undefined
    Proc./Func. XFRX_RUNMACROINAPP - Undefined
    Proc./Func. XFRX_RUNPROGRAMINAPP - Undefined
 

A file xfrxlib.fll/xfrxlib64.fll
    Unknown _XFGETLTI - Undefined
    Unknown _XFBMP2 - Undefined
    Unknown _XFCROPIMAGE - Undefined
    Unknown _XFCONVERTIMAGE - Undefined
    Unknown _XFIMAGEBPP - Undefined
    Unknown _XFWMF2IMAGE - Undefined
    Unknown _XFWID - Undefined
    Unknown _XFDEC - Undefined
    Unknown _XFIMAGES - Undefined
    Unknown _XFGETVERSION - Undefined
    Unknown _XFIMAGEGETDPI - Undefined 
 

mJindrova

RE: Project Compiler suddenly stopped including newer files

Chris,

IMHO, VFP ignore - at buildind project - all macros, EVALUATE() function and all name expression at finding file dependies.

VFP try find file:
DO PROCNAME 
DO PROCNAME IN PROCEDURES_FILE
 


VFP don't try find file (name expression):
DO ("PROCNAME")
DO PROCNAME IN ("PROCEDURES_FILE")
 


VFP don't try find file (macro):
lcc="DO PROCNAME"
&lcc.
lcc="DO PROCNAME IN PROCEDURES_FILE"
&lcc.
 

VFP don't try find file (EVALUATE()):
EVALUATE("PROCNAME()")
 

Joe,
Becase xfrx.prg contains command SET LIBRARY TO (name_expression) VFP can't find xfrlxib.fll at building project.
But (xfrx.prg) contains DO XFRX_TESTFILEINAPP WITH ... IN XFRXPROXY. In this case VFP try find XFRXPROXY.prg, but if a file can't find, then wrote messages to error log.

mJindrova

RE: Project Compiler suddenly stopped including newer files

mJindrova,

let's be totally clear: I said I use an absolute path, I have VFP installed in C:\USERS\PUBLIC\VFP9, and VFP does not include _reportlistener.vcx, if I write:

CODE

SET CLASSLIB TO C:\USERS\PUBLIC\VFP9\FFC\_REPORTLISTENER.VCX 

And that removes any cause of evaluation or anything like that. It does not work and should. The build only finds _reportlistener.vcx whemn I CD into C:\USERS\PUBLIC\VFP9\FFC and that's the current topic.

The other thing is, once you have _REPORTLISTENER.VCX in the project you can change that to as little as
SET CLASSLIB TO _REPORTLISTENER.VCX, use any non existing path and the built EXE will still find it insde the exe, if you don't exclude the VCX library. And that's two different topics.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
As a rule to thumb, I don't normally reference files outside my source directory, but it's definitely strange behavior that even referencing them with a full path doesn't just bring them in. We both tried it and it fails.

The HOME() version is for different reasons, it's dependent on where you installed VFP, so (HOME() + "ff\") in parenthesis basically is only evaluated at runtime because the compiler probably sees it as an on the fly macro substitution.

Like you, I also was thinking even the quotes around the file path cause the compiler to just tokenize it as a string that will be evaluated later when you use SET CLASSLIB TO "I don't care\_reportlistener". Even when we eliminated the spaces in typical VFP Home directories and copy the file to the source directory, it still fails. For no apparent reason.

In the end, my solution was to just copy them and even include them in my batch file that builds my distribution zip file. I know I could manually add them to the project, but since I already needed to push a bunch of FLLs, and DLLs into my batch builder, I just added a few of the other required XFRX dependencies because they're only actually called by the (ALWAYS external) XFRX.APP file, which we know now is toxic to the project compiler. So now, the external APP can always see the always external dependent files.

****
**** LESSONS I'VE LEARNED
****

1. XFRX works best external from your project as an APP or as an FXP file.

2. Referencing it inside the project can break the compiler's ability to parse the rest of your files.

3. VFP also has trouble pulling other types of files into the project like the _ReportListener.vcx. It doesn't crash it like XFRX, but it doesn't seem to include it.

4. XFRX has a ton of dependencies, including _ReportListener.vcx, some DLLs and FLLs. Once you include them in your distribution, things work fine and dandy.

I had no reason to expect the #1-3, but once we discovered it, the easiest solution still seems like #4.

I know a lot of developers use it, because it does a lot of great things, but documentation isn't clear about the potential problems when it's unusual implementation is done with the source code version.

Perhaps most people didn't pay extra for the source, and only use the precompiled version. Frankly, I'd prefer it if I knew I just needed to stick a few files in my directory and include them in the distribution. For the record, when you purchase the source version, they don't include the pre-built version, so I can't confirm what it's like using the pre-built version.

Another complication is since it's including the source for everything, including things like the DLL, you don't know which of those files are part of the required distribution itself. The answers are divided into 4 potential places, installing, compiling the source, using it in code, and distributing, without a black and white set of steps to see the big picture in one place. Perhaps mJindrova can help us with that.

So, I basically kept it external, built my normal project and added dependencies one by one to my distribution zip until I no longer got runtime errors.

I can list the changes to my zip later for those who are curious or have a better solution.

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:

The other thing is, once you have _REPORTLISTENER.VCX in the project you can change that to as little as
SET CLASSLIB TO _REPORTLISTENER.VCX, use any non existing path and the built EXE will still find it insde the exe, if you don't exclude the VCX library. And that's two different topics.
The funny thing is that files located inside an exe can always find other files inside that exe. It looks there first, then when they're not found, it looks outside, however an external file like an APP may or may not see those files for some reason even when that APP is called from the exe, which is odd.

So, another potential fix is to build the APP with those dependencies baked in. Perhaps the non-source versions do this, but I don't know because I purchased the source version and it's not pre-built.

RE: Project Compiler suddenly stopped including newer files

Joe,

thanks for confirming that.

Re 3: That's not the whole truth, because when I CD HOME() then CD ffc and then build the project, during the build the _repertlistener.vcx and _gdiplus.vcx are pulled into the project.
But if I take it as the way it ever was, then I wonder about the often recommended solution when a PJX breaks to create a new PJX, just add in main.prg and build. It would only work, if you
a) have everything in one folder
b) CD into that folder before building
or
c) alternatively to b) have all necessary PATHs in SET PATH so the files that need to be pulled in are visible from either default directory or SET("PATH"), but don't assume subdirectories aer automatically included. Nmobody hinders you to expand to as many directories including HOME() and subdirectories to not need to instead copy over source files from the VFP installation into your own project directory. It works better, if UAC doesn't sabotage things and you have VFP installed into a directory fully under your control with write permissions to compile not into VirtualStore directories.

The idea that VFP automatically pulls in files is therefore something that only works partially. Another working trick that's also often recommended is just drag a bunch of files or directories and drop them on the project manager window to let the files be added, instead of letting build discover and pull them in.

I have to say I rarely use this pull-in mechanism anyway, as said when I create a new PRG for a project I usually do it from the project and if I put in something from other projects I do use the project managers ADD and don't just wait for BUILD to pull it in.

What that tells me is that CD into the project home directory before building may give you a better experience in your compilation.

And the lesson I learn, if there is no other behavior on older Windows versions, too, is that the pull-in of files is surely not a total myth, but this has no recursive effect and does not at all work like a flood fill to complete a project just from a new empty pjx with main.prg. And, Joe, I understand by now that's also not what you're doing in general, it's still something I investigated into as you clearly state in the thread title that including new files should be automatic, when you just DO them from somewhere else that's already included in the project.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
Lots of lessons learned from this XFRX experience, especially that it was the main problem in the first place.

My main product has about 4800 files in it, with about 1400, prgs, vcx, and scx that become part of the project file. I intentionally don't include any of the FRX files in the project, because I like to be able to modify them on the fly as needed for customers.

My program evolved over 40 years, so modules are refactored and moved or deleted when I no longer use them. I generally trust that the project file is accurate and that if I write a new PRG or create a new SCX, it will end up in the project without needing to remember to do it. As a developer, I like predictability like that.

When I accidentally added the older PRG that had that SET PROCEDURE TO XFRX line, it broke reality for me because for the past few months, new features were no longer being added automatically. I'd only know they were being skipped when a customer would send me a screenshot with "xyz not found". It wouldn't fail for me in my dev environment, because the IDE always found the PRG as I ran it uncompiled. I kept adding them by hand only after getting another screenshot.

Thankfully now that it's external again, all is well. I added a new test PRG and called it, then when I built it, the project picked up the new prg, so I know all this led to unbreaking reality for me so I can focus on the program itself knowing that I can keep adding code and it will always compile without issues.

I thank you all for helping solve this mystery.

RE: Project Compiler suddenly stopped including newer files

Quote (Joe)

I added a new test PRG and called it, then when I built it, the project picked up the new prg, so I know all this led to unbreaking reality for me so I can focus on the program itself knowing that I can keep adding code and it will always compile without issues.

I still rather keep the responsibility to add in a PRG to a project before even building it. Just to be sure the EXE will also have it.

Your way will work mostly because everything is in the main project directory anyway. But you could use the PATH setting and environment manager for ensuring that things are finding their way into the project and EXE, too.

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)
I've allowed the Project Manager to add virtually every file in my project for decades because I've always kept 100% of my files in one folder and as you know, I never need to rebuild the project from scratch, but it's good to know it will work if the project file ever becomes corrupt or if I build a derived version for some reason.

I've reached a point where everything works regardless of the development machine I use. None of my code is ever stored on drive C:, which is another story, but it allows me to write code from anywhere and compile on any computer, including on-site at a client's office.

I almost always compile on a copy of code on C:, but that's a copy to a fresh folder 100% of the time. If the distribution is a "keeper", I'll archive that folder and it will be wiped before the next build.

RE: Project Compiler suddenly stopped including newer files

I have nothing to add to this thread - except to say that it is impressive that a single thread should attract over one hundred replies.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

RE: Project Compiler suddenly stopped including newer files

Haha, unfortunately you can't sort threads by most posts.

There was a lot we both went through and mJindrova deserves more stars for clearing up the core issue I never would have thought of. I know the simple source code protection is to remove PRGs and delete source code memos in SCXes, VCXEs etc., is there more protection? It's possible to use Refox to protect against refox decompilation and encrypt source code, but then you also need a decryp initialization.

I should perhaps try with the XFRX demo. It was new to me an app within a project can crash the build process, that's for sure.

Indeed trying with xfrx225-9, adding the xfrx.app from the unzipped download into a project doesn't cause a build to crash, also not with a SET PROCEDURE TO it. Either that mechanism of protection was changed or there was something else.

Chriss

RE: Project Compiler suddenly stopped including newer files

Chris,

Try add to your code next line:

CODE --> VFP

loSession = XFRX('XFRX#INIT') 

mJindrova

RE: Project Compiler suddenly stopped including newer files

Well, I did

CODE -->

Set procedure to C:\...Downloads\xfrx225-9\xfrx.app
loSession = XFRX('XFRX#INIT') 
Build still does not crash, sorry :)

Chriss

RE: Project Compiler suddenly stopped including newer files

(OP)

Quote:

Indeed trying with xfrx225-9, adding the xfrx.app from the unzipped download into a project doesn't cause a build to crash, also not with a SET PROCEDURE TO it. Either that mechanism of protection was changed or there was something else.
That sums it up entirely. In my case it was SET PROCEDURE TO XFRX. I didn't specify prg or app, but they were both present.

In the end, it's hard to imagine how any single PRG, FXP, or APP can throw a monkey wrench into the compiler.

When all the dust settled, I went back to re-read the XFRX documentation and realized that mJindrova is actually managing the documentary website, so perhaps Martina will update some of the documentation with some warnings about making sure you don't call it directly when using the compiled vs source version and what the effect will be.

Another factor is that the source version does not ship with a compiled App, so there are essentially 4 ways it could be implemented.

1. As a trail App.
2. As a paid pre-built App or FXP.
3. As paid source code integrated directly into your code.
4. As paid source code, where you compile the parts and use it as #2.

Knowing what I know now, I would've loved to just download the source and compiled versions separately, then implemented it using the instructions for a paid, pre-built app. That seems the safest and easiest thing. Since they don't compile the source version, it would be best to keep the source separate and have clear instructions of how to build the functional equivalent of #2, so all the instructions from that point would be uniform.

I think I'll ask Martin, the developer, to review this thread and consider distributing the compiled version along with the source in the future.

RE: Project Compiler suddenly stopped including newer files

Well, changing it to xfrx without .app still doesn't change it, there also is no xfrx.prg present in xfrx225-9, the only prgs including those in subfolders are locclass.PRG, source.prg, xfrxproxy.PRG,runreport.PRG, and custDet.prg.

Don't know about you, but I'm fine with the app not causing that problem anymore.

Chriss

RE: Project Compiler suddenly stopped including newer files



1. As a trail App.
It's a demo version (xfrx.app) - all outputs.

2. As a paid pre-built App or FXP.
It's a xfrx.fxp prebuilded for VFP 6+7, VFP 8 and VFP 9 (three packages) and each customer - all outputs or pdf only.

3. As paid source code integrated directly into your code.
It's a xfrx.prg.
a) You can compile for VFP 5, VFP 6, VFP 7, VFP 8, VFP 9, VFP A 32bit and VFP A 64bit.

4. As paid source code, where you compile the parts and use it as #2.
a) Package creator don't know what variant customer need - 3a).
b) XFRX can be include in destination exe/app or exclude as fxp or app.
c) Customer can build separatly xfrx.app with XFRXPreview inside xfrx.app

I prefer XFRX outside destination exe/app, because it's simpler change it.

mJindrova

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members! Already a Member? Login


Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close