Networking
Networking
(OP)
Well Im setting out on my most adventurous project yet. To incorporate network capability into a program i shall be writing.
After a pretty strong analysis of the user requirements, a networked solution would make them quite happy, even if it were to be just two computers connected together.
The thing is, I have not got a clue where to start. I guess I'd have to write a communications unit. The preferred solution is a wireless lan (probabally using a wireless adaptor card in each PC, though it may be that I have to have them connected via ethernet card and good old coax.
So my question is: Where do I start?
Of course, all help will be aknowledged in the final build as ever (not that you'll ever see the source code (I doubt you'd ask)) and will be appreciated dearly.
Thanks...
After a pretty strong analysis of the user requirements, a networked solution would make them quite happy, even if it were to be just two computers connected together.
The thing is, I have not got a clue where to start. I guess I'd have to write a communications unit. The preferred solution is a wireless lan (probabally using a wireless adaptor card in each PC, though it may be that I have to have them connected via ethernet card and good old coax.
So my question is: Where do I start?
Of course, all help will be aknowledged in the final build as ever (not that you'll ever see the source code (I doubt you'd ask)) and will be appreciated dearly.
Thanks...
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
I made a server and client application in C++ using DvNet (networking), DvThread (interface to POSIX threads) and DvUtil (various functions). These libraries are written by Dirk Vermeir (a professor at the Free University of Brussels) and are only available for Linux. You can find them here: http://tinf2.vub.ac.be/~dvermeir/software/dv/index.html
Regards,
Bert Vingerhoets
vingerhoetsbert@hotmail.com
http://student.vub.ac.be/~bvingerh/
Don't worry what people think about you. They're too busy wondering what you think about them.
RE: Networking
RE: Networking
Regards,
Bert Vingerhoets
vingerhoetsbert@hotmail.com
http://student.vub.ac.be/~bvingerh/
Don't worry what people think about you. They're too busy wondering what you think about them.
RE: Networking
Though I would like it to be maintainable so I guess the messaging would be somthing Id have to look at in depth.
So Im going with TCP/IP protocols? *searches google* Im reading up on wireless TCP IP and PCMCIA cards cause that's what Im going for.
Why am I doing it in Pascal? Its the last (big) project Ill do in Pascal and I wanna go out with a bang =D
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
If you need a direct connection, you have to let the server application set up a server socket that listens on a specified port for incoming connections. Whenever such a connection is established, you can either transfer it to another port (so that other connections can be made) or leave it at the current port (in this case you will only be able to connect one client at the time with the server).
If you only need file sharing, you can mount the remote harddrive (on which the files to be shared are located) locally to a drive letter and use this drive in Pascal as if it was a local drive (e.g. local drive = C: remote drive mounted to Z: assign(f,'Z:\file2sha.re');)
Regards,
Bert Vingerhoets
vingerhoetsbert@hotmail.com
http://student.vub.ac.be/~bvingerh/
Don't worry what people think about you. They're too busy wondering what you think about them.
RE: Networking
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
Regards,
Bert Vingerhoets
vingerhoetsbert@hotmail.com
http://student.vub.ac.be/~bvingerh/
Don't worry what people think about you. They're too busy wondering what you think about them.
RE: Networking
>Though I would like it to be maintainable so I guess the >messaging would be somthing Id have to look at in depth.
If all you need is file sharing then you don't need anything fancy like TCP/IP.
Network drives look just like any other drive, you simply read/write to them like you would a local HD. Normally you neither know nor care that they are physically elsewhere. (250,000 line application designed from the ground up to be a network app--there's *ONE* routine that actually cares where a file is. It's job is to say who has the file locked.)
What is important is file locking and how you go about updating data.
You must ensure that other computers do not mess with your data while you are in the process of changing it. The simple solution is to lock anything you are using. (Note: You'll have to implement your own lock routines. For some reason Borland neglected them.) So long as that is adequate, no big deal.
However, sometimes you can't be that aggressive about locks. An example from my code: The employee editor for our time clock. There are 3 fields directly editable when displaying the list of employees. If I were to lock the file when it was called up, though, big problems. People need to be able to clock in and out! Thus when the user attempts to change one of those fields I lock it, read it again and compare it to what I already had in memory. If they match, I write the change and unlock. If they don't match I simply unlock, redisplay what I just got from the disk and say somebody else just changed this. Editing the details of an employee leaves the whole employee's record locked because the nature of the situation is that only one person is normally doing such editing at any given time. Someone else who tries to go into the editor at that time is very likely to get a frozen screen waiting for the first person to leave.
The order scheduling program is even pickier in this regard--*EVERY* normally-edited field is protected by such a routine. There is nothing that waits for a key between the lock and the unlock.
RE: Networking
When a field is entered to be edited, it gets a locked value of true. If someone goes to the edit screen after it's been locked, they will not be able to select the field that is currently being edited.
There is also an option for the user to lock the whole record while they are working on it.
Does that sound sensible?
Thanks to all for the info about drives.
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
While it's possible to do some of your own locking (if you wish to deny specific operations but permit others) at the heart of it you *MUST* use the system-provided locking facilities. You can build upon them, you can't replace them.
When it comes to locking, "not too secure" won't cut it. It's perfect or it's a disaster waiting to happen. Always figure that the other program is watching, waiting to pounce at the most inopportune time. (This tends to be reality--when it hits a lock the OS will retry a bit, so the other program likey *IS* waiting to pounce.)
RE: Networking
When you say "the system-provided locks," what do you mean? Cause Im writing the system myself ;)
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
Example:
A travelling agency books reservations on a plane, the number of free seats (n) is an example of "shared memory", travel shops are threads.
Travel shop #1 wants to book 2 seats and checks if they are available, if so, it decreases n by 2.
Travel shop #2 also wants to book 2 seats and also checks if these seats are available, if so, it decreases n by 2.
If no locking would be used, what would happen if n=2 and the following scheme would be used:
Travel shop #1 Travel shop #2
n>=2?
n>=2?
yes
yes
n:=n-2
n:=n-2
At the end n would contain the value -2!
When implementing locks, only one thread at the time can access a certain memory location and the scheme would look like this:
Travel shop #1 Travel shop #2
lock n
lock n -> denied
n>=2?
yes
lock n -> denied
n:=n-2
unlock n
lock n
n>=2?
no
unlock n
In this case, Travel shop #1 gets the 2 seats. The order in which each undividable operation is scheduled, however, is completely arbitrarily. If Travel shop #2 would have received the first lock, they would have had the 2 seats.
This explanation illustrates the need of locking and the need of an OS that supports it. How it can be used from within a Pascal program, I don't know.
Regards,
Bert Vingerhoets
vingerhoetsbert@hotmail.com
http://student.vub.ac.be/~bvingerh/
Don't worry what people think about you. They're too busy wondering what you think about them.
RE: Networking
Loren, can you explain what an atomic operation is please? And why its used instead of my proposed method. Cheerz
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
The reason why only the OS can issue locks is that only the OS decides in which order the application chunks will be executed. Any lock handler has critical program parts that may not be split into two or more chunks; making sure this doesn't happen requires that the lock handler be embedded in the OS.
Regards,
Bert Vingerhoets
vingerhoetsbert@hotmail.com
http://student.vub.ac.be/~bvingerh/
Don't worry what people think about you. They're too busy wondering what you think about them.
RE: Networking
Thanks
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
Lock file present?
If yes, access denied, END
If not:
1.write lock file containing username.
2.Wait some time.
3.Read lock file. Your name?
If not, access denied, END
If your name:
1.Edit/write database file.
2.Close database.
3.Delete lock file.
The lock file must have the same name for different 'clients', but different names for different database files possible.
If a 'editing' client falls off, the lock file must be deleted manually.
RE: Networking
Quit trying to dodge the manual, lock it correctly. The principles are the same no matter what the language.
RE: Networking
I just wanted to include them incase, you know maintaing the system. Im running test runs with my method and Im gonna use your comments from before and see if I can come up with something special.
I get that its not going to be as good as regular off the shelf software or anything you guys make but it'll do for my proposed user.
Im also gonna look around, see if anyone does know a way to use the operating system's locks with Pascal. No doubt I wont if Bert doesnt know ;) And even if I do it'll be over my head.
Thanks for all your help
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
This is the sequence of events.
Prog 1 looks for a lock file, doesn't find one.
Prog 2 looks for a lock file, doesn't find one.
Prog 1 writes a lock file with its name
Someone asks the OS that's running prog 2 to do something big, and there's a delay.
Prog 1 reads the lock file and finds its name, and is now happy.
Prog 2 finally gets a look in and writes its lock file with its name.
Prog 2 hangs around a bit
Prog 1 meanwhile modifies the data file at will
Prog 2 reads the lock file again and still finds its name
Prog 2 starts to modify the data file at the same time as prog 1.
The difficulty is that only the operating system can decide how much processor time the two progs get, and in what order. You can't guaruntee the sequence of the thing at all. This is why file-sharing MUST be handled by the OS. The hard disk where the data are belongs exclusively to 1 OS, which means this 1 OS can control how many people are using that file, exclusively.
In fact, if the OS isn't going to do file-sharing for you, why bother having an OS? Managing files is really one of the biggest roles of the OS.
RE: Networking
Shame. Oh well. Thanks all
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
I see some discussions around, also 'real' programs like java script etc uses lock files to simulate the functionality.
Here's some 'low level' info regarding Pascal/DOS records:
http://www.ipsiamoretto.com/~azzani/borland/delphi-ti/15170.html
But thats a very OS specific, it might not work for example on a Novell server..?
RE: Networking
Regards,
Bert Vingerhoets
vingerhoetsbert@hotmail.com
http://student.vub.ac.be/~bvingerh/
Don't worry what people think about you. They're too busy wondering what you think about them.
RE: Networking
Java doesn't need to simulate locking. It uses a virtual machine (i.e. a simulated processor) with built-in multithreading capabilities, thus also locking.
Regards,
Bert Vingerhoets
vingerhoetsbert@hotmail.com
http://student.vub.ac.be/~bvingerh/
Don't worry what people think about you. They're too busy wondering what you think about them.
RE: Networking
Hmm... simulated processor ;) LOL! No dont worry Im not crazy. Ive given up with locking now but I have thought of something else.
What if the drives on different computers don't get renamed? So on the server there will be the default C
(which I guess I can change on installation) then the connected computers will all also have C:\ drives.
How do I over come the problem of reading a different C:\ drive. I could write in the user manual how to install the software and how to change the drive letter at the same time, but I doub't they'd ever use the manual.
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
In a network the server drive letter (if necessary) should be assigned another one on the clients, else the network is not configured properly. Your program doesn't have to handle anything of that.
You can pass the drive path as well as other settings to your program via the 'paramstr' function in Pascal. Then it is easy to configure and launch the software by using a .BAT file (a bit simpler than implementing a config file yourself).
RE: Networking
Don't give up on the locks--multi-user programs are *NOT* reliable without them. Period.
As for renamed drives--make the program not care. Don't store a drive letter in the first place. Put the data in a subdirectory under the directory where the program is, use ParamStr(0) to find that out. That will deal with renamed drives fine.
RE: Networking
@Every1: Thanks for the help. I think, if I find anything, I'll be adding the locks to my program pretty much last. So it gives me a fair bit of time for research.
Thanks again =D
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
Open "My Computer"
Select Tools->Map Network Drive...
Choose the drive letter to which the network drive should be mounted (e.g. "Z:")
Choose the network drive (e.g. "server\\e\database")
Now you can assign files that reside in the directory "database" (or deeper) on drive "E:" on the computer with name "server" to a Pascal variable:
assign(f,'Z:\data1.db');
reset(f);
...
I expect the network drive mapping procedure is similar under Win2000 but I don't know how it's done under Win9x.
Regards,
Bert Vingerhoets
vingerhoetsbert@hotmail.com
http://student.vub.ac.be/~bvingerh/
Don't worry what people think about you. They're too busy wondering what you think about them.
RE: Networking
Theres also a unit on SWAG so Im gonna check that out (last resort).
Actually it looks like borland have the answer:
http://community.borland.com/article/0,1410,15976,00.html
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
Regards,
Bert Vingerhoets
vingerhoetsbert@hotmail.com
http://student.vub.ac.be/~bvingerh/
Don't worry what people think about you. They're too busy wondering what you think about them.
RE: Networking
~*Gwar3k1*~
"To the pressure, everything's just like: an illusion. I'll be losing you before long..."
RE: Networking
RE: Networking
If you have a series of client programs that need to look at a database file on another computer, which will also run a program managing that file, then you could potentially do it like this:
All clients can read the database (read only)
If a client wishes to change the database, it writes a single file containing instructions about what it wants changing. This file has a unique name made up of something indicating what machine is doing the writing, and a serial number incrementing for that machine. In this way no two machines should ever generate the same file name.
The data-base computer has a "management" program that can continually look out for these instruction files. If it finds one, it reads it to check it's complete. If it isn't, it ignores it until it is.
A complete file is read, interpreted, and deleted by the management program. The management program then produces an updated database, deletes the old, and renames the new.
In this way, no two programs can ever be writing the same file at the same time. There is no chance of a client reading the database while it is being written, since it is written in another name and then renamed, and I'd imagine renaming on most operating systems is pretty much "atomic" in that no operating system worth its salt is going to allow remote file access to a file that it itself is in the middle of working on.
The worst case scenarios are (1) that a client tries to read the database between deletion and renaming. In this case it gets a simple "file not found", and can wait a few seconds and try again. (2) Two clients are trying to modify the same record. In this case it is down to the servicing program to decide which of the two conflicting instructions gets priorities.
i.e. proper locking is better!
RE: Networking
You mustn't forget that, if a lot of clients are connected, the queue of instruction files might be so long that a read operation is never sure of having the latest value; in fact, it might be a value that had actually already to be changed (according to the queue) multiple times.
Also, decoupling read and write operations is not a good idea, partly because of the aforementioned facts and partly because, from an application engineering point of view, you should keep things abstract and uniform (i.e. you should perform read operations with an instruction file too).
Another weak point is the timestamping method: in stead of using a client-specific timestamp, you should use a global timestamp that increments with each operation.
A lot of the mentioned problems flow forth from the fact that a message passing system with files is used. A direct network connection from client to server would be a better idea. But then again, Pascal might not be the best language to write this sort of application in.
You also could write a user-friendly interface to a mySQL client and use a mySQL server to deal with datafiles.
Regards,
Bert Vingerhoets
vingerhoetsbert@hotmail.com
http://student.vub.ac.be/~bvingerh/
Don't worry what people think about you. They're too busy wondering what you think about them.
RE: Networking
The main weakness of a system like this, as you are totally right to point out, is that there is a delay between asking for something to be done, and its happening, so during that delay, other processes will be given out-of-date information. That's bad, very bad, but sadly not entirely unknown in lots of aspects of life....
RE: Networking
Regards,
Bert Vingerhoets
vingerhoetsbert@hotmail.com
http://student.vub.ac.be/~bvingerh/
Don't worry what people think about you. They're too busy wondering what you think about them.