Archive for the '3D Web' Category

How to Keep Physics Engine Load Low

What is ODE?

Today most OpenSim installations use Open Dynamics Engine “ODE” as physics engine. It is delivered as integrated part of current OpenSim versions. It is used to simulate a (more or less) realistic physical environment in the virtual world. The physical behavior of physical objects, like some vehicle types, and avatars is calculated by ODE.

Under certain circumstances the physics engine can be very busy and that physics engine load can be experiences as server lag. This article tries to describe how to identify physics engine lag and how to improve your regions to cause less such lag.

Additionally to the server lag you experience much physics engine activity might cause that OpenSim consumes more and more memory over time, due to memory leaks. This may cause lower stability and more frequent restarts of your OpenSim installation.

How to identify physics engine lag?

Enter “show stats” on your OpenSim console window. After that you will see some output like this:

ASSET STATISTICS
Asset cache contains        0 assets
Latest asset request time after cache miss: 0s
Blocked client requests for missing textures: 0
Asset service request failures: 0

CONNECTION STATISTICS
Abnormal client thread terminations: 0

INVENTORY STATISTICS
Initial inventory caching failures: 0

FRAME STATISTICS
Dilatn  SimFPS  PhyFPS  AgntUp  RootAg  ChldAg  Prims   AtvPrm  AtvScr  ScrLPS
  1.00      55    45.5     0.0       0       0    3516       0     419      14

PktsIn  PktOut  PendDl  PendUl  UnackB  TotlFt  NetFt   PhysFt  OthrFt  AgntFt  ImgsFt
    27      10       0       0       0     5.3     0.0     4.3     0.0     0.0     0.0

MEMORY STATISTICS
Allocated to OpenSim : 152 MB

Have a look at the value shown under the label PhysFt. This number is the so called Physics Frame Time. If that value is high, above 20 or even higher,  your physics engine is quite busy. Best run “show stats” multiple times and check if you always get more or less the same PhysFt values or if you see peaks of physical engine activity.

If you see such peaks, even if nobody is currently visiting your region (use “show users” command on the OpenSim console to check that), then you probably have moving objects causing physics engine lag. Otherwise it is possible that moving avatars cause such lag, for example by walking on prim surfaces or by colliding with other objects or avatars.

The PhyFPS value shows the Physics Engine Frame Rate. Values above 45 fps are good. Usually high PhysFt values cause low PhyFPS frame rate values.

You can also display the Physics Engine Frame Rate in the viewer. Select the menu item “View > Statistics Bar”. The Statistics windows opens and shows the Physics FPS value under the label Simulator. If you cannot see it, simply click on the label Simulator.

Reasons for physics engine lag

In the following situations the physics engine has to do calculations:

  • Movements of physical objects
    • unscripted physical objects (i.e. a ball)
    • scripted physical objects (i.e. vehicles implemented as physical objects)
  • Movements of avatars
    • avatar walking over terrain
    • avatar walking over a prim surface (normal prims or sculpties)
  • Collisions between non phantom objects and avatars
    • avatars colliding with each other
    • non phantom objects colliding
    • avatar colliding with a non phantom object

Many physical movements require the detection of collisions. Collisions are detected based on so called Bounding Boxes, rectangular boxes using the X, Y and Z axis and that contain the whole object, avatar or segment of terrain.

You can toggle displaying these internal Bounding Boxes in the viewer using the following menu item in the Advanced menu (which can you open using ctrl alt D):  Advanced > Rendering > Info Displays > BBoxes

Bounding Boxes

How to reduce physics engine lag?

To reduce physics engine lag you should do the following:

  • Try to avoid physical objects (i.e. use vehicles that are not physical objects)
  • Never create huge physical objects, because they easily cause many collision events
  • Make objects phantom if possible (collision detection ignores phantom objects)
  • Make moving/rotating objects phantom
  • Use llTargetOmega and llSetTextureAnim for viewer side rotations/animations
  • Do not use hollow prims with the intention to let other objects or avatars move inside
  • Do not use too complex, rough sculpty prims and terrains as walking surface for avatars

The general rule is to try to avoid collisions of Bounding Boxes as much as possible to reduce physics engine load.

By the way, that is also the reason why avatar attachments are always phantom objects.

One of my renters could reduce Physics Engine Frame Rate values from about 50 with peaks up to 150 to much lower values of about 4. The main reason was a hollow prim with a scripted vehicle constantly moving inside. This did cause physics engine lag. But a bigger issue was the constantly growing memory consumption (memory leak) and thus more frequent region restarts than necessary.

Appendix

This is a short description of the more cryptic labels of the “show stats” console output:

  • Dilatn time dilation
  • SimFPS sim FPS
  • PhyFPS physics FPS
  • AgntUp # of agent updates
  • RootAg # of root agents
  • ChldAg # of child agents
  • Prims # of total prims
  • AtvPrm # of active prims
  • AtvScr # of active scripts
  • ScrLPS # of script lines per second
  • PktsIn # of in packets per second
  • PktOut # of out packets per second
  • PendDl # of pending downloads
  • PendUl # of pending uploads
  • UnackB # of unacknowledged bytes
  • TotlFt total frame time
  • NetFt net frame time
  • PhysFt physics frame time
  • OthrFt other frame time
  • AgntFt agent frame time
  • ImgsFt image frame time

Snoopy Pfeffer
Founder and CEO of Dreamland Metaverse

snoopy.pfeffer@yahoo.com
http://www.3dmetaverse.com/

PayPal Money Module



After Adam Frisby did announce his DTL PayPal Money Module for OpenSim about a month ago, which got much positive feedback, I did test it intensively and I did extend it’s functionality. Last week Adam and I did decide, that it is the best if I continue to work on it using a fork of his GIT repository.

You can find that GIT repository with the latest version of the PayPal money module for OpenSim here:  http://github.com/SnoopyPfeffer/Mod-PayPal This fork has been renamed from DTL-PayPal to Mod-PayPal to avoid confusion.

Functional Extensions and Bug Fixes

  • User2User pay, User2Object pay, User2Object purchases and User2Land purchases work properly, now
  • User2User payments are confirmed with instant messages to senders and receivers
  • Support for group owned objects and land can be enabled (default: off); requirement: PayPal email accounts have to be defined per group
  • Underscores in email addresses do not cause error messages during OpenSim startup anymore
  • Object and land purchases for US$ 0 do not initiate a PayPal transaction anymore
  • Object purchases properly transfer the purchased contents to the buyer, now
  • User email addresses are loaded from the local OpenSim.ini file; invalid email adresses are ignored, but loading is not aborted anymore
  • User email addresses can also be fetched from the Users grid service, if that feature is enabled (default: off); fetched email addresses are cached by the region server until region restart; local email addresses always take precedence, to ensure an acceptable level of security
  • Locally defined user email addresses should be used for all users that receive bigger amounts of money within a region (i.e. shop owners); the reason is the much higher security, if these email addresses are not fetched from the Users grid service; beside that it is possible to locally use a special PalPal micropayment account for all shop transactions to save fees, instead of a standard PayPal account used elsewhere
  • Group email addresses can be defined per group UUID in the local ini file to support group owned objects and land as mentioned before
  • Non existent user or group email addresses are handled by showing warning messages, instead of error messages for crashed code
  • Added start and success log messages for all kinds of PayPal transactions
  • Rounding problems made it impossible to pay certain amounts (like US$ 1.23); now all amounts work and are displayed nicely

Interesting findings

  • PayPal can send money to all email accounts, even to email accounts not registered at PayPal yet; thus it is not necessary to handle that case
  • PayPal micropayment accounts are not available in all countries worldwide

To Do List

  • Web pages that are shown after successful or cancelled PayPal transactions
  • Locking for PayPal transactions while buying land or original objects
  • Events or functions that allow to lock vendors while a PayPal transaction is in progress
  • Too small amounts that do not cover the PayPal transaction fees produce a misleading error message on the PayPal web page; maybe it would be good to be able to define a minimum amount globally and/or per email address to avoid that
  • It would be good if a scripter using llGiveMoney gets a warning message, that this function is not supported by the PayPal money module
  • User2User payments are confirmed with instant messages to the sender and receiver of money, but only if these users are online at that time; maybe it is possible to improve that, so that even users offline get such confirmation messages as stored offline messages

Installation of the PayPal Money Module

  • Install the addon-modules/mod-paypal under OpenSim/Region/OptionalModules/
  • Install the files-for-bin-dir in the OpenSim bin folder
  • Add the additional config settings in the config-include subfolder or in the OpenSim.ini file in your bin folder
  • After that compile OpenSim

If you find bugs or if you have ideas for the future development of this module, please send me an email (snoopy.pfeffer@yahoo.com). Thank!

The Costs of Free Resources

We all like it: When we get something for free!

But the non existence of costs has also disadvantages: We tend to ignore the resources that are associated with what we use. This is simply the case, because there are no costs for us, that otherwise would provide an incentive to make a more economical use of resources.

In OSGrid users like that it is possible to upload textures, animations and sound files for free. Additionally the users do not have to pay for activities like creating new groups. To make it clear: I fully back these policies, because they support the development of OSGrid.

And at this point we should all  thank the donators that cover the costs of the central infrastructure of OSGrid and that help keep it running so well every day. For sure you all know about the new “donate” button on the new Elgg based OSGrid web site.

But the missing incentives to save resources has negative effects, that over time increase the costs for central grid services unnecessarily, reduce the performance and lessen usability in certain areas.

In the following paragraphs I pick two examples to describe these issues and how these could be solved by implementing policies and tools. With this article I want to initiate a discussion about these issues, before we experience a future rapid grows of free Opensim grids.

A)  Asset Usage

People tend to upload textures, animations and sound files even if they exist within the world already. From the user point of view the costs – the time needed to check if for example a texture is available already – is higher than simply uploading it once more. Such duplicates inflate the space required for the asset database and asset caches. Additionally duplicates increase the data that needs to be transferred and thus reduces Opensim performance.

Additionally people tend to upload multiple versions of an asset to test it in world until they are happy with the result. At the end they just use the last version. Such unused assets, assets that are not referenced anymore, still consume space in the asset database that could be freed.

Possible Solution

One solution would be to introduce incentives for a more economical use of these resources. Second Life does this with their L$10 uploading fee.

But even better would be, if Opensim is able find duplicates and unused assets. Alternatively this can be done on the fly, by using some Opensim extension, or by using database scripts that are run regularly, as part of an asset database maintenance task. The goal is to remove all unreferenced assets and to replace duplicate assets with a single copy, which might include the replacement of the corresponding UUIDs.

To efficiently find duplicates, data block sizes and hash values can be used. For sure only assets that cannot be changed by individual users separately can be replaced by a single asset copy. This is especially true for uploaded textures, animations and sound files, that cannot be changed within the viewer. Notecards and scripts, that can be changed in world, and other changeable assets cannot be replaced by a single asset database copy.

In practice OSGrid uses Fragstore to do dupe compression based on 256 bit hash values for nearly 3 million assets, now. The compression achieved is about 25 percent. Currently no deletion of unused assets is done. The detection of unused assets seems to be impossible or at least very difficult with the current asset server architecture Opensim uses.

B) Creating New Groups

Groups have been introduced with Opensim revision 9215, but already today there exist nearly 600 groups in OSGrid. I have had a closer look at the existing groups and saw the following:

  • Most groups just have one member
  • There are even many groups with no members
  • Some groups have very similar names and it looks like that most of them have already been given up for newer groups with a slightly improved title

If the development continues like this, we soon will see many thousand groups, where it is difficult to identify which groups are trash and which are still used. For certain keywords group search already returns many groups with many similar names (search for “star” for example).

Possible Solution

To prevent an exaggerate use  of groups, that also reduces usability, policies and tools that support these policies should be used.

In Second Life people have to pay a fee to create a new group. Additionally the group owner has to recruit more group members within 3 days, otherwise the group gets deleted. And a group must maintain a membership of at least two members all the time in order to remain active.

In OSGrid I think we do not need a fee, but a policy for active groups similar like in Second Life. This means that empty groups and groups with less than 2 members would get deleted at latest 3 days after creation. This is the case for most groups that currently exist in OSGrid.

Additionally it might be useful to check when all group members have logged in the last time. If no group member has logged in during the last 90 days, it can be assumed, that all group members are not active OSGrid residents anymore.

A database script could check all existing groups regularly and delete groups that do not fulfill the previously mentioned conditions. For sure Opensim needs to behaves properly, if a group is encountered that was deleted and if such a group is used by land or objects shared by or deeded to that group.

In the case of shared objects and land, the group defined should simply be removed, making it unshared objects or land again. Deeded objects and land is more tricky, because in that case the group also needs to be deleted and the owner has to be reset to the previous owner before the object or land was deeded. All this should just happen if that group really was deleted. A disabled group module (maybe by accident) should not reset all group memberships.

Well, the last part was a bit technical, but it shows that implementing such a policy is something that needs to be well thought through.

Finally a little history story: As we all know, teleporting between regions also uses resources, especially processing power and bandwidth to transfer data. That is why during Second Life “stone age” they tried to introduce a fee for teleporting between regions. But understandably it did not take long, until users were demanding to abolish this teleporting fee.

I am sure that we will find a solution to prevent an exaggerate use ressources with policies and tools, without having to use money as incentive.

Group Permissions for Land and Objects

A while ago, mcortez and his flotsam project have provided Opensim the source code to add group functionality to Opensim. And after a short interruption, because of security issues that needed to get fixed, it is now part of all newer Opensim versions and available at most OSGrid regions.

The Groups Module allows people to define and join groups. So today we all can use fancy group tags like in Second Life, which are defined as part of the group roles. In addition it is possible to send and receive group instance messages and group notices. Voice conferences can be used if all attendees are on voice enabled regions using the same voice server.

What was still missing were group permissions for land and objects. These allow land and object owners to define the rights other people have to change the contents of their parcels of land or to change objects itself.

This functionality is essential for collaborative building projects, where you want to allow just a specified group of builders to be able to rez or change objects. Till now, you had to allow all people to build on your land and to modify common objects (by tagging “allow anyone to move”). This was quite risky, because griefers could rez objects or even destroy what you have built together.

With today’s Opensim trunk version 9817 group permissions for land and objects have been added to Opensim.

To get group permissions working, I had to add group permission checks to the Opensim Permissions Module. This new functionality is very similar like in Second Life, but currently there are still limitations, because some related functionality has not been implemented in Opensim yet.

For example it is not possible to define object inventory items as being shared. And currently it is not possible to allow group members to edit certain settings of land they do not own in the About Land window. Thus currently it is not wise to deed land to a group, but land shared by a group, with the corresponding rights set in About Land, is a very useful new feature.

Where to get Support for Opensim

Support for Opensim

These web pages describe where you can get support for Opensim:

The best sources for help are the Q&A Sessions each Saturday in OSGrid, the IRC channels #osgrid and #opensim, and the mailing list opensim-users.

For sure many people that you will meet in OSGrid, on the IRC channels or via the mailing lists like to help you to get your own Opensim regions running in OSGrid.

Finally you can find the latest OSGrid announcements on Twitter at http://twitter.com/osgrid, including planned down times, required Opensim updates and other urgent news for OSGrid users.

Samsara

Contact

I hope that this blog helps many people! Comments how to improve these articles and requests for additional topics to cover in the future are welcome. Additional articles will be added over time.

Snoopy Pfeffer
OSGrid and Second Life resident
snoopy.pfeffer@yahoo.com

p.s. I also offer professional services for Opensim and 3D Metaverse development for corporations. This includes strategic consulting, business development, marketing in 3D worlds, edutainment solutions, content creation and IT advisory services.

OSGrid – Why and How to Join

Why to join OSGrid?

Since over a half year I run a couple of regions in OSGrid. OSGrid is the biggest Open Source Metaverse that currently exists (see http://osgrid.org/). I have chosen this virtual world, because it allows people to run their own regions using Opensim (http://www.opensimulator.org/), a 3D application server for virtual worlds, and because it is the main Opensim grid. It always uses and tests the latest versions of this new, quickly evolving technology.

From the beginning I was fully aware, that Opensim is still under development (alpha software) and that especially in OSGrid, using the latest versions, this means that you need a kind of pioneer spirit. On one hand that means that I can use the newest features and bug fixes of Opensim very early. But on the other hand, new bugs or fundamental protocol changes can cause temporary problems and seldom even service disruptions.

For someone who runs own region servers in OSGrid, it is normal to regularly spend time to test new Opensim versions, beside improving the own Opensim installation and the processes for service management.

The main reason why I have decided to use Opensim already today, probably still about 6 months before it will become beta software, is that the functionality and stability is quite impressive already. Beside that I want to support the development of Opensim as a technology and to help to build up a community of people within OSGrid already today.

The community within OSGrid is quite impressive: Beside software developers you meet old Second Life pioneers, great content developers and residents of all ages enjoying to explore this new frontier of technology together. Compared to Second Life there is a quite relaxed spirit and willingness to help each other. Many inhabitants like to share the great things they have built with the other people in OSGrid.

OSGrid

The Open Metaverse of the Future

Without doubt, the development of an Open Metaverse, based on highly distributed 3D servers, is a technology that will enable the next evolutionary step of the World Wide Web by integrating information and communications in a much more natural way than everything we have seen before.

The integration of information and communications in a a human friendly 3D environment, where switching between 2D and 3D views is possible at any time and where each person has a global identity, user profile, friend lists, interest group memberships, a virtual inventory, simplified payments, etc. will add much value to every persons life as well as for the companies offering products and services in this future 3D Web.

I believe, that Opensim has the potential to become the basis technology of what will become this future 3D Internet. Till this vision becomes reality, there is still a long way to go. First features that were found useful in existing virtual worlds need to be fully implemented. Then the next step will be the integration of the traditional World Wide Web and real world applications, creating new means of information sharing, communication and interaction of people using this future 3D Web. This required combined development efforts on the 3D viewer and server side.

But now let’s go back from visions to already very concrete developments of Opensim and OSGrid.

My first impressions in OSGrid

When I have joined OSGrid a year ago, I saw, that beside important features still missing, that most people who had signed up did not like their newbie “Ruth” look. It gave them the impression, that Opensim was still not ready for real use. And at that time there was no possibility, beside uploading or creating your own clothes, skins, etc. to improve your look. That was the reason why I have started the shopping area at Samsara. Samsara offers donated free skins, shapes, eyes, hair, clothes, etc., as well as shops with textures, plants and tools for builders.

Homestead land versus renting or running whole regions

My next impression was, that many people are interested in owning land in OSGrid. But many did not have their own Opensim server running yet. Or they were even not interested in running their own Opensim server at all. At the beginning most people show interest in small parcels of land called “homesteads“. Homestead land parcels allow people to start building and to get a first impression. Many residents of OSGrid offer free homestead parcels to people new in OSGrid. I have also started to offer homestead parcels on my 4 tropical atoll regions South of the Samsara shopping area.

Other people like to have their own full regions. But for that you need to install and run Opensim yourself, unless you find someone who is willing to run a region server for you. I saw that there is a demand for Opensim regions to rent, but on the other hand that there is no bigger professional company offering such services in OSGrid yet. The reason is, that maintenance and support costs for alpha software are quite high.

I have decided to offer such regions in OSGrid already today, based on a price that just covers my costs. I do this, because I think it would be bad for the development of Opensim, if non IT specialists would be locked out. Many people do not care about the details of the technology used (installing, configuring and running Opensim, updates, etc.). They just want to use Opensim and they have very valuable feedback for the future development of Opensim. My regions use the newest recommended Opensim releases and provide enhanced Opensim features like groups, offline messaging and voice.

What does it mean to run your own Opensim server?

Anyway many people who join OSGrid quickly ask, how they can get their own Opensim region server running on their own computer at home. I have the impression that most underestimate the complexity of installing, configuring and running Opensim, although it is clearly stressed that Opensim is alpha software and still under development. For most people the troubles already start with the required network setup, that often causes most difficulties at the very beginning.

Before you decide to run your own region server, you should be aware of the following:

  • Opensim is alpha software and still under development. Do not expect software that you just install and it works.
  • The documentation for Opensim is still limited and sometimes it takes some time to find the required information.
  • You will invest a reasonable amount of time in maintaining your regions and testing problems you have encountered.
  • You need to have at least basic software development and network management skills.
  • Expect that you will identify software bugs that you should report using a tool called Mantis (http://opensimulator.org/mantis/).
  • Support the continuing development of Opensim as a tester and by proposing possible improvements.
  • If you encounter problems with your Opensim installation, good debugging skills are very helpful.
  • If you connect to a grid like OSGrid, expect that you have to update your Opensim installation regularly.
  • Often you will need to update to Opensim trunk versions for which no executable files exist yet.
  • You should know how to compile software from source files on your computer (especially Mono and Opensim).
  • You need to be able to set up your local network so that people can access it from the Internet.
  • If you have a dynamic IP address you need to setup a service like DYNDNS (http://www.dyndns.org/).

If you have the skills previously listen and if you are willing to take the burden of using software that is still under development, you can join and support a really important open source project that probably will shape the future of virtual worlds and even the future of the Internet.

The purpose of this blog

In the past many people have asked me, if I could write a blog about how to install and run Opensim on Linux servers. Finally I have found time to start this blog. – Sorry for the long delay!

My goal is, to collect all information pieces that are widely distributed at the moment and to add information that even is not documented yet, like how to use Monit for Opensim service management. I want to provide a step by step guide for people interested in setting up their own Opensim server on a Linux server (I use Ubuntu).

Here I will not repeat what is documented somewhere else already. I will just reference such documents. Instead I will try to give an overview how all parts fit together. I will describe how to get an Opensim region server up and running, how to integrate it in a grid like OSGrid and how to maintain the service in an efficient way. This blog intends to provide additional information wherever this might be useful.

In general the main sources of information for these topics are http://opensimulator.org/, see section “Running your own OpenSimulator”, and http://osgrid.org/, see “Instructions > Attach Region”.


Blog Stats

  • 21,588 hits

Pages

July 2017
M T W T F S S
« Jun    
 12
3456789
10111213141516
17181920212223
24252627282930
31