I want to mirror a directory over an internet WAN link - any ideas?

[xxpaulcpxx]

I'm trying to think of a system for my Dad's company. He's in the country with little broadband access. I've showed him my satelite Inet connection, but PC Anywhere SUCKED over it - way to much latency for his tastes. He needs to share files seemlesly accross the organization. I was thinking of software that could mirror a directory accros the WAN link to multiple sites, and keep track of latest versions/prevent overwrites. I think Notes isn't the correct solution (overkill, perhaps?), just a simple file based system would be better.

Anybody have a way to connect to a shared dirtectory but access local copies of the files, while maintaining file integrity?

[kickdrum]

Why not just set it up like a WAN? Just enable file sharing on your TCP/IP connection and create short-cut's to the directory you want access to? Such as:
\\XXX.XXX.XXX.XXX\Directory
You just have know the IP one of the Computers. Probably the Satellite connection since the IP is static. The files wouldn't be local but I don't see how you could do that because if the data changes it would have to be updated every so often. Maybe "My Breifcase" would work over the WAN type setup. Good Luck

------------------
Kickdrum
jameymarrs@home.com

[xxpaulcpxx]

I looked up My Briefcase in help, and found something better in Win 2K. It's called Offline file shareing, and windows will update a local copy of a network file or directory on a log on/off basis or continuously - exactly what I'm looking for. Anyone familiar with this option?

[twwabw]

Paul-

Offline folders may well be a good solution for him. BUT- understand it before you implement it! Since you used to use pcAnywhere, think of it as similar to PCA's "synchronize folders" option.... sort of. What really happens, is that a copy of your network stored data\folders\files\whatever, is copied locally to your laptop's hard drive(assuming that's what it is).

This is not a substitute for a WAN connection- it merely means you can locally access data while not attached to the network. The when you return, and hook into the work LAN, the changes are updated either to, or from your laptop, according to chich version changed- the server stored copy, or your local copy. But beware-

Let's say there was an accounting Excel file he sync'd on Tuesday when he left the office. We'll call it revision "a". The next day, someone in accounting makes a change to the file. That's rev b. Now, while your dad's out of the office, thursday he decides he'll work on the spreadsheet too, and saves his changes. Now, when he returns to the office on friday, and plugs in- which version do you think will be kept?

Your Dad's. Good for your Dad- bad for the accounting dept.

This solution is best used for personal data, or data that would never be shared among others in the organization for their use or editing. If the NTFS share was set to read only, that would be OK I guess.

But, what exactly are you trying to do? I think you're actually trying to set up a VPN via your satellite. Isn't your satellite connected via phone line for upstream communication? All of this sounds like you're trying to set up subnets across the internet.... if you are, don't bother. VPN could be a solution for sure, but don't try to set up file shares to servers across the INet. There is more to this than you could imagine, and you can't do it for a multitude of reasons.

Tim

[xxpaulcpxx]

Ah - But the offline folders would be syncronized, over the WAN, and refreshed every few minutes. There are only about 10 people who would be accessing files MAX, and only a couple would want the same file at the same time (possibly). Files are updated rarely, but read often. After the initial replication of the shared files, traffic would be minimal - especially since the copies are local - not WAN as in a VPN setup. The VPN provides the link, but Offline Files brings the content to local locations.

That's the theory anyway. Comments?

[twwabw]

Paul-

What WAN?? Do they have a WAN already between multiple sites? A true WAN, w/shared resources, replication, etc.? If they do, what do you need this for? I guess that's the part I don't understand.

Offline folders would never be refreshed, unless and until they were accessed- there is not any traffic generated until then. There is no replication every few minutes.

You mention you want to prevent overwrites, but that is just what this feature does- it overwrites the original server copy if the file is changed. You also say several people have access to these files. Bad idea if they can change them.

Your last sentence doesn't make sense to me- " The VPN provides the link, but Offline Files brings the content to local locations". A VPN also brings access to the files to the locations- that's the whole idea of it- a virtual extension of the network to other users\nodes\sites\etc. Transmission of data, ie: files, would only occur of they were accessed.

"a way to connect to a shared dirtectory but access local copies of the files, while maintaining file integrity" - this is a mouthful, Paul. If you're talking about one person's access- fine. Many people accessing- not so good, imho. If WAN links already exist, why would this be necessary?

Please explain further.

Tim
MCSE, MCP+I

[xxpaulcpxx]

OK, lets run this from the top:
EXISTING- 1 main location, 7 remote sites.
EXISTING- all users need to view and update various MS files from a master server.
PLANNED- satelite broadband Inet access at remote sites (500-800 kbs bandwidth, high latency, Starband doesn't use phone lines - it's a new system).
PLANNED- VPN connection to main site to create the WAN connection over the Inet.
PLANNED- create offline file shareing to distribute and update files accross the network.

You were asking why I'd want to offline share files when I already have an always on VPN connection. Easy response - speed. Which would you rather open, a 1mb file on your local drive, or a 1mb file on a drive that is access though your 500-800 kbs network connection? Local, of course! Offline file shareing will bring those files to your local system AND propagate changes.

We use offline shareing, but never go offline. The files are refreshed - by which I mean checked for changes against the main copies on the remote server - at intervals of 1 minute (this is done via Syncronization Manager). This constant checking should maintain the latest information at all locations. Overwritting may be a problem, but one I'm sure can be delt with easily - their main concern is usability. Their main usability issue is performance.

Quoting two things from you Tim:
"Transmission of data, ie: files, would only occur of they were accessed." and "If WAN links already exist, why would this be necessary?"

To reply to your statement and answer your question: We want the files transmitted BEFORE they are accessed. The WAN provides the access, but it is too slow.

[twwabw]

Aaaahhhh- now I begin to see the whole story... it wasn't clear to me what actually already exists in place. I'm assuming that more conventional communications are not available, due to location? It was also my initial impression that we were talking about single users at the remotes, not entire LANS. Sorry.

No, I am not familiar at all with Starband. Are upload and download speeds both 5-800kbps? This not a slow connection speed at all- many of our WAN links function quite well across 128 & 256 fractional T-1's. And that includes Exchange traffic, as well as network traffic, and all sites have many users running thin client accounting packages via Citrix Metaframe. And all of this via 2 and 4 port Frac T-1's.

In any case, since folder and file permissions as well as certain aspects of caching can be set , perhaps offline files may be a good solution for you.

Just curious- what kind of data files are these? Word documents? Excel? Databases? If so, I wonder about using thin client software for your application. It is certainly a low overhead process. The actual file being manipulated is only on the server. Your clients then are in escence sent a screen only, with only keystrokes and mouse clicks transmitted over the WAN. Its other advantage is absolute data integrity- a local file is never kept, and is never even transmitted- you just utilize its data over the link. Figure a min of 8-10K of the pipe per user, which can be as high as 20K ea. if there are frequent screen refreshes.

[DurocShark]

Look at the Cobalt Qubes. This is what they're designed for. :-)

[twwabw]

Originally posted by DurocShark:
Look at the Cobalt Qubes. This is what they're designed for. :-)

I've never seen these, so I just went to their website. Are you sure you mean the Qubes, or the CacheRaQ? They sure look interesting.

Do you use these devices? Pros / cons? Tell us about them.

[xxpaulcpxx]

You are correct about location limiting the availability of DSL, and fractional T1 would be cost prohibative (we're talking about locations in the middle of the countyside). They do have DSL access at one location, and they are very happy running PC Anywhere over that, they wish they could do similar at their other locations.

I understand what you are saying about Citrix solutions, however, these look very expensive as well, and they don't have a full time IT staff to figure out its management. They would be running it over a 26kbs phone line!

The main problem with using the Starband systems for any kind of remote control software is the high latency factor. When testing the remote control capabilities, the 780 kbs download capacity (150 to 300 upstream) looked EXACTLY like the 26 kbs phone connected session I had been shown. The only difference was that the screen refreshed in larger chuncks, but too longer to do than the smaller chunks the 26 kbs system did that came slightly faster.

You mentioned "sending the screen" as a solution - which PC Anywhere does quite well, especially over low bandwidth mediums. Our bandwidth is not the issue, it's the time it takes to get that screen t the client PC. In a sense, I have a dumptruck vs a hand shovel and I have to fill a hole. With the hand shovel, I have to take lots of small scoops, but the pile I'm scooping from is close to he hole I'm filling, so it will take me 30 minutes to fill the hole. With the dumptruck, I load up all the dirt in one scoop, but I have to drive down the access road onto the interstate, take a shortcut through a residential neigborhood, pull up to the hole and dump it all in. That whole trip took me 30 minutes. It's not the load that I can carry that is the issue (as we are mostly used to experiencing), it is the time it takes to get the data anywhere. This is not as much an issue with web browsing - which Starband excells at.

Online gaming and remote control software both need low latency connections, otherwise the interface is useless. If it takes you 3 second to know where your mouse click actually ended up - you will shortly put your foot through a keyboard!

I'll take a look at a straight VPN solution for them though, though I think they'd like the mirrored directories better. Their previous VPN's were at 26 K, mabey it's time to look at them again!

As for the Cubes, they would be great if it was only LANs we were going to, but I see home use being important as well - and thos would be pretty pricy iron for home use. I think a software rather than hardware solution would still be best for this case.

Keep these Ideas coming, we'll figure out something!

[tolken]

you speaking mirror or image? if the latter I suggest GHOST

[twwabw]

Paul-

I hope I'm not beating this to death, but when I glanced by the thread again, I looked in to see if you'd found a solution. As I keep re-reading this, there are some things that keep popping up.

First, this may just be a case of symantics, but this scenario as I see it is NOT a WAN. Unless you're planning on putting PDC's, BDC's, setting up WINS, etc., etc., you're not creating a WAN. All you're doing here is a VPN extension of your LAN at best. That's OK, because it will probably still do what you need it to do. But, that's still not the real problem.

As I have looked into CSC, it appears this could work for you. It allows your users, or locations, to map a physical drive to the server folder(s) / file(s), and perform the synchronization automatically in the background.

But you never did say what kind of files these were, and I see that as the biggest potential drawback. How are you going to account for the fact that multiple users could be accessing the same file at the same time, and making changes on it? Especially when the last to save will be the one who's changes are kept? This would not be the case in a database file, where it is meant to be multiuser accessed, and only the individual records change. But what are you going to do about things like spreadsheets? How are you ever going to control this? If you're only going to have changed implemented on the main location server side, and you just want the clients to see the changed versions, this would be fine.

I'm curious to know how you make out with this.

Tim

http://home.rochester.rr.com/twwonline/files/syncmgr.gif

[xxpaulcpxx]

Well, everything is still pretty theoretical at this point. I'm waiting for the guys at their end to load up NT so we can use RAS for a secure connection.

As to the file changes issue, here is the plan. All the files are excel, word, and an access database. The files will be centrally located on a main server. All the remote sites will locally replicate from that central site.

If the central site makes a change, it will flow out to the remotes, overwriting what is there. If the remotes make a change, it is propagated back to the central - assuming the central is not currently being changed. The file that was changed at one remote site will be written to all the remote sites after it has been placed on the central site.

If two people are changing a file at the same time, then there will obviously be a replication conflict, and I'm not sure how MS will handle this. I am also not sure if/how MS may propagate the information that a file is currently being accessed.

I do agree that they need to move fully to a database - unfortunatly they also need someone inhouse to manage it as they always want to do something new with their info. I know - Access is easy - it's more a matter of someone having the full time to understand it and not hose their existing database. I did this for them about 6 years ago, and they are still using my database. Unchanged. OI!

ADDED THOUGHT: I just reread your latest post and got an idea. If all the files on the central site were read-only, all the changes at the remotes would have to be new files added to the central directory. No file conflics. Very cludgy, but simple enough that even sales/men-women might understand!

[xxpaulcpxx]

I've done a limited test here at my home with the sync feature, and I'm pretty impressed. If there is a change to a file before you are able to save it back, it will allow you several options - including creating a copy with your userID, time and date appended to the filename. It will also alow you to set defaults for the overwrite protections to! I'm setting up the VPN next week, we'll see how it goes.

[twwabw]

I'd really like to hear how this works out for you. We're considering this as part of an upgrade strategy at an Architectural firm I do work for, as part of the W2K laptop upgrade process we are thinking about.

Let us know!