Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": No module named svn). Look in the Trac log for more information.

Ticket #1151 (closed enhancement: migrated)

Opened 8 years ago

Last modified 3 years ago

VisitManagers should delete expired entries in tg_visit

Reported by: grover Owned by: anonymous
Priority: normal Milestone: 1.5
Component: Identity Version: 1.0b1
Severity: normal Keywords: visit data housekeeping
Cc:

Description

Otherwise tg_visit will just keep all its stale visit entries forever. It would also be nice if when an entry in tg_visit was deleted, its corresponding entry in VisitIdentity? was deleted too.

Attachments

delete_visits_from_db.diff Download (1.3 KB) - added by gkalab 7 years ago.
Patch to delete visit entries that are expired from the database (only for SA yet). Diffed against 1.0 branch.

Change History

comment:1 Changed 8 years ago by jorge.vargas

  • Milestone set to 1.0

comment:2 Changed 8 years ago by alberto

  • Milestone changed from 1.0 to 1.1

comment:3 Changed 7 years ago by alberto

  • Milestone changed from 1.1 to __unclassified__

Batch moved into unclassified from 1.1 to properly track progress on the later

Changed 7 years ago by gkalab

Patch to delete visit entries that are expired from the database (only for SA yet). Diffed against 1.0 branch.

comment:4 Changed 7 years ago by gkalab

I use a background process to access TG every few seconds. Visit entries are created on these anonymous requests. So the database was growing rapidly and I had the need to delete expired visits from the database.

The suggested patch implements this for SQLAlchemy only and it is only tested with PostgreSQL. With PostgreSQL stale entries in the visit_identity table can also be deleted easily with the proposed patch and the following table setup:

visits_table = Table("visit", metadata,
    Column("visit_key", String(40), primary_key=True),
    Column("created", DateTime, nullable=False, default=datetime.datetime.now),
    Column("expiry", DateTime, index=True)
)

visit_identity_table = Table("visit_identity", metadata,
    Column("visit_key", String(40), ForeignKey("visit.visit_key", ondelete="CASCADE"),
           primary_key=True),
    Column("user_id", Integer, ForeignKey("tg_user.user_id"), index=True)
)

comment:5 Changed 6 years ago by Chris Arndt

  • Keywords visit data housekeeping added
  • Milestone changed from __unclassified__ to 1.6

While the issue is still valid, the approach taken by the patch does not seem a good one for me. Checking for expired visits in the VisitFilter would mean that this would be done at every request, which will most probably impact performance.

I suggest that the cleanup procedure is called in the VisitManager.run method, like it is done with update_queued_visits.

Then, of course we need a SQLObject implementation and tests, before this can be applied.

comment:6 Changed 6 years ago by faide

  • Milestone changed from 1.6 to 1.5

comment:7 Changed 6 years ago by chrisz

Right, the VisitManager seems to be the appropriate place for deleting expired visits.

In one project I'm also dealing with many users created on the fly from a remote LDAP repository. In this case, in addition to the visit table, expired entries can also accumulate in the visit_identity and tg_user tables. Maybe we should also provide a solution for that.

As a temporary solution, I have added an example in  http://docs.turbogears.org/1.0/Scheduler that shows how you clean up the entries using the scheduler.

comment:8 Changed 5 years ago by Chris Arndt

For the time being, I added a recipe for a function to clean out old visits to the doc wiki:

 http://docs.turbogears.org/1.0/UsingVisitFramework#housekeeping

It's not the most effective one, but I wanted to keep the example simple for the sake of clarity.

comment:9 Changed 4 years ago by chrisz

  • Type changed from defect to enhancement

comment:10 Changed 3 years ago by chrisz

  • Status changed from new to closed
  • Resolution set to migrated
Note: See TracTickets for help on using tickets.