PacketFlow - For Ashita And Windower

言語: JP EN DE FR
2010-06-21
New Items
users online
フォーラム » Windower » News » PacketFlow - For Ashita and Windower
PacketFlow - For Ashita and Windower
First Page 2 3 4 5 6 7 8
Offline
Posts: 1157
By DaneBlood 2022-07-07 16:54:56  
IBHalliwell said: »
I would change "true" to "false," correct?
That is true
[+]
 Asura.Aburaage
Offline
サーバ: Asura
Game: FFXI
Posts: 266
By Asura.Aburaage 2022-07-07 19:56:27  
For some reason when I change it to false, it keeps reverting back to true
 Fenrir.Positron
Offline
サーバ: Fenrir
Game: FFXI
user: Positron
Posts: 181
By Fenrir.Positron 2022-07-07 21:58:03  
Is it located in Program Files or another privileged directory? You may need to launch your text editor as an Administrator to save changes to the file.
necroskull Necro Bump Detected! [58 days between previous and next post]
 Gilgamesh.Briga
Offline
サーバ: Gilgamesh
Game: FFXI
user: Modron
Posts: 1
By Gilgamesh.Briga 2022-09-03 17:13:04  
Fenrir.Positron said: »
Is it located in Program Files or another privileged directory? You may need to launch your text editor as an Administrator to save changes to the file.

/windower/plugins/settings/packetflow.xml
necroskull Necro Bump Detected! [47 days between previous and next post]
 Leviathan.Wiccaan
Offline
サーバ: Leviathan
Game: FFXI
user: Wiccaan
Posts: 27
By Leviathan.Wiccaan 2022-10-20 16:24:51  
Hey everyone, firstly sorry for the bump. I don't come on this site much at all so I haven't kept up with the post here regarding our plugin. I was linked to the thread last night in regards to a question about implementing something similar in another game so I got a chance to read the replies. (Edit: For those that don't know who I am, I am atom0s. I am the lead dev of Ashita. Forgot the forum has my old characters in-game name.)

Wanted to say thank you to the kind words a lot of you said towards me. It's appreciated. :)

I also wanted to give some of my input on the whole SE hiring and my personal experience/information with the game to reply to some of the things said here.

Asura.Geriond said: »
Does this also help with input lag (IE, you issue a command and it does go through, but it takes like 1-2+ seconds to initiate)?

Input lag in the sense of lag that is happening entirely client sided (prior to sending or receiving any packets to 'execute' an action) is generally the result of poor frame times (not to be confused with FPS). Frame time is the amount of time it takes between each frame to render the next. In FFXI's case of being, by default, locked at 30fps, this means that you have 33.333ms for each individual frame of that 30 to fully handle all background tasks (updating all entities positions, updating each entities event VM states, processing all input, processing the current packet queue, etc.) If you start to go over that amount of time per frame, you will see spikes in your frame time which will result in the feel of input lag. Even though you can have these spikes and feel the input lag you can still be locked at 30fps and think something else is wrong.

If you are on a laptop running Windows 10 or newer, it is recommended that you use a translation layer proxy that will up-port the game from Direct3D8 to 9/10/11/12. (Or to another API like OpenGL/Vulkan) You will see a massive improvement in performance, especially on Nvidia hardware, because Direct3D8 will not cause Nvidia Optimus to trigger and make use of your dedicated GPU hardware. And while it may seem odd, the better / newer your laptop is, the worse XI will run on it because of this.

Other things you can do to help this is:

Ashita/Windower/Stock XI Options: Lower your mipmapping setting, reduce your background resolution, ensure your background resolution is square (ie. 4096x4096), when possible run your menu resolution the same as your window resolution to remove additional scaling math, use the lower end graphics settings for things like fonts, textures, maps, etc.

In-Game Options: Disable in-game shadows, disable weather effects, lower your clipping plane.

If you do not use a controller, it is HIGHLY recommended you disable it in the gamepad config tool for XI as well. FFXI constantly polls for a controller every 2-3 seconds. This was fine back in the day when computers had very few devices connected to it and prior to every device connected now having like 8 sub-devices for each added feature (extra USB ports, RGB lighting, profiles, built-in memory, etc.) Due to this, you can see a ton of micro-stutters or straight-up pauses every few seconds due to the game looking for a controller.

In some cases with things like Corsair keyboards, your game may even fully freeze for up to 1-2 minutes causing you to DC cause of this bug. I can go into more detail as to what causes this if people are interested.


Asura.Iamaman said: »
I'd expect the server is smart enough to discard any packets that are abnormal or don't pass basic parsing. I also think Thorny mentioned position packets earlier, my assumption being these are typically high frequency, small size packets that would not raise alarm (I'd also assume both server/client have some form of keepalive, but maybe position fills that role also)

This has been a fix-it-as-it-becomes-a-problem situation for FFXI. The game was coded with pretty poor security practices in place (for both the game and PlayOnline for that matter) so packet protections have literally been an as-needed type of fix. For example, back in the day, you could easily crash an entire zone by sending a malformed chat packet that had half an auto-translate tag without a proper opening/closing. The server has trusted the client a lot more than it should over the years, which is why there has been dozens of dupes found throughout the years with little effort. This is pretty common for MMOs though as security oftentimes lands up being a 'do it later' goal and that goal never gets done until it has to be from an exploit. For the most part, they have added extra layers of checks and protections to the packet system on the server side, but there are still holes in it.


Asura.Specialkid said: »
Kinda feels like that skeleton crew of a skeleton crew just doesn't know how to do modern programming imo

we can do all this stuff with external tools fixing the problems from the outside in, but these guys are hardfocused on fixing what they can from the inside out (which is not a bad idea, but that requires NOT being on a skeleton crew of a skeleton crew)

XI isn't really modern programming tbh. It's coded in C/C++ from the 90's era of coding and still uses old tooling that was from the PS2/PC cross-over development era. (It's compiled with VC6 still.) SE has said in previous interviews/Q&A's that the teams are/were still using PS2 dev tooling to work on the game for the PC version. (But I'd expect by now they have made some additional tooling to make PC dev easier, at least for content development.)

However, I also don't feel its fair to expect the small team they have working on XI to push out fairly big changes to the core code at this point. It's not as easy as just editing a few lines of code and going live. Changes to the core/engine of a game takes a lot of time and costs to fully cover things to avoid any kind of massive outage to the game if things go wrong. At this point in XI's life, they just don't have that kind of team size and budget put into modifications to the actual core.

For reference, all the changes that have happened to the client over the last handful of years have been strictly things that are not 'breaking' in a manner that would yield the game unplayable. They are more of 'safe additions', basically copy-pasting existing working code to add new things. (ie. Master level system.) Or just adding more onto existing systems. (ie. Extending wardrobes further.) These kinds of changes aren't game-breaking if something goes wrong and can be easily patched in an emergency maintenance.

SE tends to avoid going the route that will require rollbacks. If they implement a big change to the core/engine and that change is wrong/bad, it could lead to a massive amount of bad data being generated in the database which leads to even more problems. Which they don't have the team/budget setup for this game anymore.

XI makes plenty of money to pay a team, they just choose not to do it at this time still.


Odin.Deridjian said: »
Matsui-P should get an intensive full-time English course to at least B2 to maybe C1 as part of a professional company training and speak to some persons of the Windower/Ashita community, then push to Yoshi-P and the CEO's to hire them freelance for a while. Can't even imagine what some people could accomplish if they just had the real back end on their hands. UX would be fixed in no time. You guys doing so much work around this game, it's astonishing. Great job actually posting this. Much love to all the devs and data miners out there!

I don't personally agree with this. There are a couple of things that would be a problem and a conflict of interest.

Firstly, in regards to SE hiring us (Ashita/Windower devs), this could come at the cost of our projects. Having to sign NDA's and additional agreements to work on XI's source could leave us in a position where we can no longer work on Ashita or Windower, or any similar/related/connected projects. This would require either us to negotiate in some manner to allow the projects to continue and let us still work on them, or something.

Next, this could be seen as a negative thing for SE and affect their other games / future projects and past decisions on how they handled bans. If they hire us, it sets an example that they are ok, to some degree, with the modding community of a game they have been very anti-third-party tooling over in the past. It is very fair and easy to say that XI's development has been shaped by our tools, but it's not as easy for SE to openly say 'ok' or agree to them, vs. being silent on the matter.

As for the UX overhaul comment, I don't think you understand just how in-depth XI is designed around its UI and menu system. This isn't a simple get-hired and overhaul in a month type thing. Sure people can adjust some stuff to clean things up and redo some textures and some other changes, but fully overhauling would be a literal full-on project itself. (I am not talking about hiding the current UI and adding custom things in place, as that has/is already being done, that isn't really the same thing as fully overhauling.) A good majority of the game's code is intertwined with the menu system.


Asura.Saevel said: »
Likely they wouldn't be able to accomplish much if anything. POL / FFXI is old and was designed and built using old technologies and techniques, they might not even be using x86-compatible hardware for the servers. The entire reason we can do stuff with FFXI now is that both Windower and Ashita teams have built a framework that enables them to go around the client. They aren't attempting to fix or modify ancient C code (I think that's what the windows client is written in), opting instead to figure out where it's various functions are kept in memory and just hook into them directly.

SE isn't full of "lazy terrible developers", those guys work their butts off to produce whatever they can. In all likely hood they don't even have anyone around who even knows whatever screwball language the server engine is written in, much less is familiar enough with the networking libraries to not cause catastrophic damage changing them. Pretty much everything they are doing nowadays is scripting / object editing or resource modification, things that can be done with specialized in-house tools and don't require editing engine code.

As I mentioned above in another response, XI is coded in C/C++ using VC6. While Ashita/Windower are both coded in C++ as well, we use newer compilers/tooling. Parts of the language have changed since the VC6 era, but nothing to an extent that any experienced C++ developer wouldn't be able to work with. The client and servers XI uses are all x86 code. I can't say for sure what additional technologies are being used on the server end of things though. (ie. if they use any kind of scripting or additional things for the game servers.)

On the POL side of things, that's already [mostly] all reversed and being worked on privately. POL itself runs on 4 servers (patch, auth, profile, web). I outlined a bit of information in regards to those here: https://www.reddit.com/r/ffxi/comments/v4gjve/things_i_have_found_out_about_the_playonline/ib6g9yi/


Asura.Arico said: »
There are literally thousands of people who specialize in doing this exact kind of thing. It's not that hard. Sure, maybe At0mos can't do it(for the record, I would be surprised if he couldn't) but there are definitely lots of people they could hire that could in probably months if not weeks go over the entire codebase and fully document it assuming it's not documented.

Not to say this as a brag or anything, but I have most of the client reversed and documented, in some manner, already without the source. It's not really needed to get an understanding of how the game operates and functions. I'm pretty familiar with XI's inner core workings along with POL's. While seeing the actual code will still require time to get familiar with, myself and some others that work on this game would have an easier time getting situated vs. someone completely new imo. (Please understand I don't hold myself in any higher regard/standard than anyone else, there are plenty of capable people that work on XI.)


Asura.Skyekitty said: »
Is there a way to make a packet that somehow represents multiple packets so more information can be pushed through the same "hole"? Kinda like a zip file packet?

Just asking, I know nothing of this stuff.

XI already operates this way. Packets are sent in chunks which contain multiple packets 'stacked' together. The client buffers packets to send to the server and then pushes chunks (compressed + encrypted) within the current rate limit and up to a max packet buffer size. The client and server both have systems to prioritize certain data, along with 'overwriting' some data that is being duplicated within the same packet frame. (For example, the client queue packet command has additional arguments that tell it if a packet already exists if the new one should be ignored or overwrite the old one before its sent.)


RadialArcana said: »
So a question about this addon, can the developers just put systems in place on the server to ignore whatever this addon does and just block it?

If they wanted to, yes. But it would require modifications to the server and client to be able to ignore it properly/fully.
The plugin abuses something the client and server are already designed to do.


Quetzalcoatl.Xilkk said: »
next pie int he sky is to ask a rebuild of ffxi on ue5 :P

There are already people working on various client rework projects. Each one at varying stages of implementation. Along with reworks of other parts of the game (ie. fully emulating PlayOnline). Each of these projects are discussed elsewhere and not really posted about publicly outside of more specific communities.

UE/Unity implementation of the client has been a thing for a pretty long time with various people working on it a hobby project. In the last few years, a more collective push to work on something as a group has been formed but their progress is slow and just still a side project for most of those involved.

A C++ reimplementation is being worked on as well.

As for PlayOnline emulation, that is something I am working on in my spare/hobby time when I'm bored. I've shared some information about it on my Patreon as well as on a few Discord servers, but since it's a hobby project I opt to not full-on advertise it currently since I'm working on it alone. (This kind of project though opens up a bit more than just a fun thing or for private servers, but instead also can be implemented as a replacement to current POL client for retail, allowing for a handful of different things.)
[+]
Offline
By Draylo 2022-10-20 17:15:55  
I had to stop using it on my laptop, kept crashing when I enter instances and start doing stuff. Couldn't figure out why because it was only happening on my laptop and not my PC. I would go into Lilith, summon fenrir and start doing corsair rolls and every time it would have this really strange super lag, and then an instance would crash. If I kept trying to do more stuff without clicking "ok" to close the crashed instance, another instance would crash. Weird but after unloading this it worked fine again.
Offline
Posts: 4619
By RadialArcana 2022-10-20 18:15:23  
I wish someone could create something to store mipmaps on the hard drive, a cache or something. It sucks that XI creates them on the fly as you walk around. Limits what you can do with HD modding and also causes problems with multiboxers.

If you multibox, setting mipmapping to 0 on alts (keep it to about 2-4 on main to stop shimmering) should stop those fps drops/freezes when you run across a load of people or monsters.
[+]
 Asura.Cluste
Offline
サーバ: Asura
Game: FFXI
user: Bjggi
Posts: 15
By Asura.Cluste 2022-10-26 07:30:11  
Thanks to everyone who worked on it, a gamechanger for multiboxers.

Has Sortie been added to the instance ID's? Had few problems with the upper floor, but the lower one lags from time to time again.

I could turn instancesOnly off, but the "Trade" Addon from Ivaar, doesn't work anymore if PFlow is on.
 Fenrir.Niflheim
VIP
Offline
サーバ: Fenrir
Game: FFXI
user: Tesahade
Posts: 749
By Fenrir.Niflheim 2022-10-26 13:38:34  
Asura.Cluste said: »
Thanks to everyone who worked on it, a gamechanger for multiboxers.

Has Sortie been added to the instance ID's? Had few problems with the upper floor, but the lower one lags from time to time again.

I could turn instancesOnly off, but the "Trade" Addon from Ivaar, doesn't work anymore if PFlow is on.
Byrth added Outer Ra'Kaznar [U2] 17 days ago, but i am not sure if it is on live yet. the version with Outer Ra'Kaznar [U2] is 0.0.0.5
[+]
 Bismarck.Nickeny
Offline
サーバ: Bismarck
Game: FFXI
user: Nickeny
Posts: 2252
By Bismarck.Nickeny 2022-10-26 14:04:32  
Byrth, If you ever come to NY - I have a blunt and a bottle of Hennessy waiting for you

And tacos
[+]
 Asura.Sechs
Offline
サーバ: Asura
Game: FFXI
user: Akumasama
Posts: 10133
By Asura.Sechs 2022-11-08 09:01:47  
Lately I changed the only setting in the settings.xml from "true" to "false".
I'm not sure if it's purely coincidental, but it seems to me that since then Packetflow stopped working when I'm in Sheol?

I start see monsters "jumping" around instead of moving, frequently having my JAs activating like 5 seconds after I press the macro, taking forever to sometimes engage/disengage/switch and occasionally see everybody (monsters and party players) completely stop for a few seconds before my client updates again.


Now... This could depend entirely on my ISP being very bad with SE servers lately and having nothing to do with the change I did to packetflow but just in case is there anything I can do to test things out?

Like purposedly delete the settings.xml so that a new, default one will be created the next time I load Packetflow?
 Asura.Seizan
Offline
サーバ: Asura
Game: FFXI
user: Seiza
Posts: 124
By Asura.Seizan 2022-11-08 09:38:34  
Asura.Sechs said: »
Lately I changed the only setting in the settings.xml from "true" to "false".
I'm not sure if it's purely coincidental, but it seems to me that since then Packetflow stopped working when I'm in Sheol?

I have had it on "false" sinse day one. No issue here, but the one thing that i cant ty SE for enjoying the game again.
I havnt tryed dDyna yet, but all other zones is just smooth with minimum to zero lag.
 Lakshmi.Byrth
VIP
Offline
サーバ: Lakshmi
Game: FFXI
user: Byrthnoth
Posts: 6187
By Lakshmi.Byrth 2022-11-08 17:31:47  
Asura.Sechs said: »
Lately I changed the only setting in the settings.xml from "true" to "false".
I'm not sure if it's purely coincidental, but it seems to me that since then Packetflow stopped working when I'm in Sheol?

Right click your plugin, choose "Properties" and then click the "Details" tab. If it says 0.0.0.5, then it should be active in Sheol regardless your settings.
[+]
 Asura.Sechs
Offline
サーバ: Asura
Game: FFXI
user: Akumasama
Posts: 10133
By Asura.Sechs 2022-11-08 17:39:24  
I'm good then, it must be a temporary issue with my ISP and SE servers (and the network in between) of the last week-ish.

Thanks Byrth!
 Asura.Geriond
Offline
サーバ: Asura
Game: FFXI
user: Gerion
Posts: 3185
By Asura.Geriond 2022-12-03 10:23:43  
Does Packetflow affect how fast Bazaars, Auction House lists, and Wide Scan lists populate?
 Lakshmi.Byrth
VIP
Offline
サーバ: Lakshmi
Game: FFXI
user: Byrthnoth
Posts: 6187
By Lakshmi.Byrth 2022-12-03 11:55:04  
When you have non-instanced zones enabled, yes for bazaars and widescan but maybe not AH lists.
Offline
By Draylo 2022-12-04 19:00:47  
Any idea what could cause that crash when entering instances and starting to buff?
necroskull Necro Bump Detected! [31 days between previous and next post]
 Asura.Seizan
Offline
サーバ: Asura
Game: FFXI
user: Seiza
Posts: 124
By Asura.Seizan 2023-01-04 10:37:52  
Draylo said: »
Any idea what could cause that crash when entering instances and starting to buff?

Are u getting an error message?
I have been running this fine for long time w/o any crashes, but latly i have been crashing after zoning, specialy when zoning out at end of a sortie run.
Then i get an Microfost Visual C++ runtime error!
Can be somethin else, but i have a hunch that its related to PF ...
necroskull Necro Bump Detected! [156 days between previous and next post]
 Carbuncle.Elvaanmoq
Guildwork Premium
Offline
サーバ: Carbuncle
Game: FFXI
user: Imoq
Posts: 217
By Carbuncle.Elvaanmoq 2023-06-09 07:56:18  
(Apologies for the necro bump).

I just started using this yesterday as a fix for Odyssey really bad lag on a new computer (haven't tried Dyna-D yet, Sortie, Omen and Lilith were OK). It works beautifully! Haven't done an extensive testing but I really like what I've seen in the last 24 hours so far.

I wanted to report that I've had a couple of crashes that may (or may not) be related to starting using this PF. First one zoning out of an instance (after a Lilith fight), as described by the person above but also a couple more while entering the instance (Lilith fight, again) and while calling the trusts and buffing so pretty much also in line with the report previous to the last one.

Is this "a thing" or perhaps we are just having different issues? I can't imagine this being a known problem and nobody else reporting it but just wanted to make sure.

Again, my main goal for this post was to say THANK YOU for developing something like this, but also to see if my issues may be related to it.
 Shiva.Thorny
Offline
サーバ: Shiva
Game: FFXI
user: Rairin
Posts: 2850
By Shiva.Thorny 2023-06-09 08:05:30  
It is certainly possible that crashes are related, this plugin only tweaks the frequency with which packets are exchanged. On most computers and installs, this seems perfectly stable and hasn't caused any problems. But, any crash would be tangental (not caused by packetflow itself faulting, but caused by a failure of the client or pc to properly handle something at the rate it occurs with packetflow enabled), so if you feel the crashes are being caused by packetflow you'll likely not receive any help with them.
 Carbuncle.Elvaanmoq
Guildwork Premium
Offline
サーバ: Carbuncle
Game: FFXI
user: Imoq
Posts: 217
By Carbuncle.Elvaanmoq 2023-06-09 09:16:32  
That is fine, I can live with the occassional crash (IF it was related) and keep benefiting from the way more playable situation.

Appreciate your response and the effort the devs put in here.
Offline
Posts: 4619
By RadialArcana 2023-06-09 09:56:07  
Just turn the addon off, and only urn it on when you know you're doing something that needs it?
[+]
Offline
Posts: 4619
By RadialArcana 2023-06-27 09:32:39  
posting this video since it's useful to people on this topic no doubt

YouTube Video Placeholder


they talk about the the new server virtualization systems at 57 mins, which allowed them to use old operating systems and all other stuff
[+]
 Shiva.Thorny
Offline
サーバ: Shiva
Game: FFXI
user: Rairin
Posts: 2850
By Shiva.Thorny 2023-06-27 10:40:56  
Not really any technical information shared there, I am not sure if the translation is just lacking, but it's a very vague overview. Virtualization is very standard across most service hosting, but without more details it's very hard to understand anything meaningful about the change.

From their description, I believe they are running an older operating system on their virtualized machines, because their server software isn't compatible with an OS that supports the capacity of current machines. However, it is certainly possible they have some elements of proprietary translation occuring, which would explain the 2 year process of migrating it. If not, this is not a job that should have taken them any meaningful amount of time, definitely not years.

The question about data limits was likely a translation from byrth or my original question about the data transfer rates, and they brushed it off by talking about player limits per server. Without understanding the native language, it is difficult to be sure, but that was my impression.

When they talk about data issues, seeing their head of information systems(i believe that's Nakata's role?) say 'there is a lot of data on the internet overall, and that is impacting things as well' is absolutely insane and almost insulting. While they do admit that their server processing is the bottleneck(though as we showed with this plugin, it is not actually a processing issue, but some sort of buffer limit issue), they immediately move past it to make those absurd claims.

My impression from watching is that this is intended solely as a puff interview to give some credit and motivation to Nakata's team. There isn't any dense information provided, none of what they said has any value to third party devs. However, I only watched from around 0:52 to 1:15, fwiw.
[+]
necroskull Necro Bump Detected! [115 days between previous and next post]
 Carbuncle.Snicky
Offline
サーバ: Carbuncle
Game: FFXI
user: Snicky
Posts: 6
By Carbuncle.Snicky 2023-10-20 11:29:29  
I know this isn't a proper bug report and probably not terribly useful, but...

The first time I tried this add on I enabled it in Dynamis Jeuno (D).

At some point my chat log became corrupted with the last message just being
"(S"

and when I tried to open the full chat log the game hung.

I've never seen that sort of crash before so I'm assuming it was PF, but who knows...

Single boxing, Windows 10, no gearswap or anything that does stuff automatically for me. If there is any information I can provide, let me know.
necroskull Necro Bump Detected! [31 days between previous and next post]
By crazy00 2023-11-20 00:41:51  
where do you get this adodon? i tried googling it and nothing came up.
 
Offline
Posts:
By 2023-11-20 01:18:11
 Undelete | Edit  | Link | 引用 | 返事
 
Post deleted by User.
necroskull Necro Bump Detected! [222 days between previous and next post]
Offline
Posts: 49
By smii 2024-06-28 17:53:56  
any bans yet?
VIP
Offline
Posts: 787
By Lili 2024-06-29 05:35:12  
smii said: »
any bans yet?

You're now banned from asking if anyone was banned. Just you specifically, others can ask.
[+]
Offline
Posts: 23
By ShepardGarrett 2024-06-29 11:11:05  
I love Radiohead, and childish Gambino, while not a band per say I also enjoy.
I love a lot of the classic stuff to like rolling stones and skyward, and orchestra music.

I also appreciate keeping items grouped together and palletized for transporting, and to prevent shifting.

Also their a good way to solidify a wedding or for folks of like minds to come together.
First Page 2 3 4 5 6 7 8
Log in to post.