Page Index Toggle Pages: [1] 2 
Topic Tools
Hot Topic (More than 10 Replies) Add once-daily "digest" option for notifications (Read 5,466 times)
cepheid
Senior Member
****
Offline



Posts: 516
Add once-daily "digest" option for notifications
Jun 5th, 2009 at 12:27am
Post Tools
Right now, the only way that users can receive notifications is one email per topic and/or per PM... or not at all.  There is no middle-ground.  For users who are subscribed to a lot of active threads, that could lead to many messages per day.  Additionally, it means many more emails sent out from the forum, which could be a problem for admins whose ISPs like to place limits on their email volume.  (I know that email volume is currently limited by providing only a single notification per topic until the next visit, but that can still be many emails.)

It would be nice if there were an option, both for Admins and Users (see below for the difference), to send only a single "digest" email once per day (at a pre-determined time) that contains links to all threads updated since the last email or since last login, whichever was more recent.  That digest could also contain a summary of all new PMs.

The benefits of this feature for the admin would be that if he wants notifications on but to reduce the amount of email sent out by the server, he can enable only the Digest mode, which guarantees that no more than one email per member will be sent out each day.  The Admin can also ensure that these notifications are sent during off-peak hours, if desired.  If the admin chooses to enable both Digest and Individual modes, the user also benefits from having more control over the amount of email he receives from the forum (and possibly when).

The detraction: because the forum software only runs when accessed, a cron job will be needed to guarantee that the digest emails are sent out at the chosen time; if the Admin allows users to choose their own time, this could require a large number of cron jobs (if the time choices are restricted to on-the-hour, this can be limited to 24, but that's still a lot).  Otherwise, without a cron job, the chosen time is merely the trigger, i.e. the forum will send out the digest emails upon the first access after the chosen time - for forums which receive very little traffic, this could be quite a bit after the chosen time.  In practice, however, any forum exposed to the internet will be hit by a search engine, spider, spammer, or person at least once every few minutes, so digest emails should be sent out relatively close to the chosen time even without a cron job.

The implementation:

For Admins:
In the Admin Center, the Admin could either: allow digests in addition to the current system, restrict it to ONLY digests, or turn notifications off entirely.  If the Admin provides both methods, the choice is available to the user (see below).  This setting would apply both to threads and PMs; if "digest" is selected and PM notifications are enabled, they would be included in the digest.  The Admin could also select a pre-determined time to send the digest (e.g. midnight), or allow users to select their own time.

If this is implemented using a cron job to send the emails exactly at the predetermined time, it is recommended to NOT allow users to select their own time, so that a single cron job can handle it.

The interface: in the Admin Center, instead of the current checkbox to enable/disable thread notifications, there would be a drop-down menu with the options of "None," "Digest only," and "Individuals or digest."  This would replace the need for 3 checkboxes, otherwise.  The Admin could also select a pre-determined time (via a text box or series of drop-down menus) when digests will be sent; a checkbox allows the Admin to determine whether this time is fixed or whether it is merely a default but the user is allowed to select their own time.  No additional settings or changes are needed for the interface related to PMs.

For Users:
If the Admin has enabled both Digest and Individual modes, the user could select which mode to use, or none.  If the Admin has enabled only Digest mode, the user can select that mode or nothing.  If the user selects Digest mode and has enabled PM notifications, the PM notifications are included in the digest.  If the user selects Digest mode and if the Admin allows the user to pick the email time, the user can change the default time here.

The interface: in the Options, replace the current yes/no drop-down for email notifications using a drop-down with options for "None," "Digest," or "Individual."  If the Admin has disabled Individual mode, it would not be included in the drop-down.  If the Admin has allowed users to change the default notification time for digests, a text box or drop-down menu (the same as used in the Admin interface) is included here.  No further changes are needed to the interface related to PMs.
« Last Edit: Jun 5th, 2009 at 12:31am by cepheid »  
Back to top
WWW  
IP Logged
 
Corey Chapman
YaBB Administrator
*****
Offline



Posts: 10,024
Location: Rock Hill, South Carolina

None
Re: Add once-daily "digest" option for notifications
Reply #1 - Jun 5th, 2009 at 5:26am
Post Tools
Agreed - digest mode is useful.
  

Back to top
IP Logged
 
deti
Legacy Dev Team
Development Team
****
Offline



Posts: 2,650
Location: Prien am Chiemsee, Germany
Re: Add once-daily "digest" option for notifications
Reply #2 - Jun 5th, 2009 at 11:13am
Post Tools
Nice and useful idea. I see two main problems:

- Not every user has CRON-job access.

- If you do it over the forum be called some when, we will have to check on every call if it is time to send emails now and then, if there are many emails to be send at one time, the user calling YaBB.pl might have to wait a long time until his request is answered entirely because the processor might be sending a lot of emails. This could even more become a problem if the amount of emails on big forums is so high that the server runs out of time before all email are send and the user request is answered. Not talking about impatient users going away or hitting other links meanwhile the email sending process is running.

If you find a good solution to avoid both things I would be glad to hear. Smiley
  

Was immer Du tun kannst
oder erträumst tun zu können,
beginne es.
Kühnheit besitzt Genie,
Macht und magische Kraft.
Beginne es jetzt.
Whatever you can do
or dream you can,
begin it.
Boldness has genius,
power and magic in it.
Begin it now.
J. W. Goethe
Back to top
WWW  
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Add once-daily "digest" option for notifications
Reply #3 - Jun 5th, 2009 at 11:31am
Post Tools
deti wrote on Jun 5th, 2009 at 11:13am:
Not every user has CRON-job access.

Right, I knew this would be an issue, which is why I also suggested that it could be done by the script... that means the time isn't guaranteed, though, since it would have to happen only on script access.  We could even forego cron access entirely, just for simplicity, though if a user has it, I think we should take advantage of it.

deti wrote on Jun 5th, 2009 at 11:13am:
If you do it over the forum be called some when, we will have to check on every call

The time-check itself isn't a major issue - the script already does much more processing on each call, so adding a quick time-check shouldn't be a major issue.  But:

deti wrote on Jun 5th, 2009 at 11:13am:
if there are many emails to be send at one time, the user calling YaBB.pl might have to wait a long time until his request is answered entirely because the processor might be sending a lot of emails.

Yes, quite true.  But, this can be avoided by serving the user request first, and only processing the emails after the user request has completed.  From the user perspective, the request is complete once YaBB has closed stdout (i.e. has finished sending its HTML output).  Once YaBB closes stdout, the user is no longer in the picture, and the script is free to do whatever extra processing is needed.  Therefore, if you handle sending the email only at the very end, after the user request has finished, then the user should never even notice.  (And since a new instance of the script is called every time someone loads a page, there won't be any delays for any other users, either.)

The only downside (whether you use cron or the script) is that the CPU itself could get bogged down if there are a lot of emails being sent, and this can be avoided by having the script send emails out in batches, e.g. splitting up the emails so 10% (or whatever) gets sent out every few minutes until they're all done.  This would only be an issue on XXL forums running on slow machines, though.
  
Back to top
WWW  
IP Logged
 
deti
Legacy Dev Team
Development Team
****
Offline



Posts: 2,650
Location: Prien am Chiemsee, Germany
Re: Add once-daily "digest" option for notifications
Reply #4 - Jun 5th, 2009 at 11:39am
Post Tools
Good idea, we send the user request first and then let the process run for lets say 15 seconds. Then stop it and continue only when the next one clicks on the page. We must only prevent that two calls on the page at the exactly same time don't make the server send all emails twice (once by each process).
« Last Edit: Jun 5th, 2009 at 11:39am by deti »  

Was immer Du tun kannst
oder erträumst tun zu können,
beginne es.
Kühnheit besitzt Genie,
Macht und magische Kraft.
Beginne es jetzt.
Whatever you can do
or dream you can,
begin it.
Boldness has genius,
power and magic in it.
Begin it now.
J. W. Goethe
Back to top
WWW  
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Add once-daily "digest" option for notifications
Reply #5 - Jun 5th, 2009 at 11:47am
Post Tools
deti wrote on Jun 5th, 2009 at 11:39am:
let the process run for lets say 15 seconds. Then stop it and continue only when the next one clicks on the page.

I think this can be selectable by the Admin (in terms of whether to enable load management or not), and I wouldn't necessarily limit it by time but by the number of emails.  Most Admins will probably be fine letting all emails be sent at once; the load issue would only be a real problem on slow servers with super-large forums.

deti wrote on Jun 5th, 2009 at 11:39am:
We must only prevent that two calls on the page at the exactly same time don't make the server send all emails twice (once by each process).

I don't know that you'll ever have two calls at exactly the same time, but you could have a call come in before the other call is finished, yes.  But, you can prevent duplication by using a lockfile - if it's time to process, the script sets a lockfile, runs the process, and deletes the lockfile when done.  If another instance of the script is called in that time and thinks it's also time to process, it will check for the lockfile and will skip the process if the lockfile is present.  This way, only a single email process is going on at any given time.

(You only need to enable the lockfile during the email processing, not on every call.)
  
Back to top
WWW  
IP Logged
 
Unilat
Development Team
Theme Team
****
Offline



Posts: 1,047
Location: Columbus Ohio, USA
Re: Add once-daily "digest" option for notifications
Reply #6 - Jun 5th, 2009 at 2:06pm
Post Tools
Just use AJAX to send a request out to the server. The server will run it and no client will have to wait for results.
  
Back to top
 
IP Logged
 
OH Eng
Past Team Members
Documentation Team
Offline



Posts: 4,026
Location: Pensacola, Florida USA
Re: Add once-daily "digest" option for notifications
Reply #7 - Jun 5th, 2009 at 4:40pm
Post Tools
Quote:
I think this can be selectable by the Admin (in terms of whether to enable load management or not), and I wouldn't necessarily limit it by time but by the number of emails.  Most Admins will probably be fine letting all emails be sent at once; the load issue would only be a real problem on slow servers with super-large forums.


Admins have no idea how many emails/hr will ever be sent.  To know that, they would need to know what every member has set for notifications, and how many topics will be posted in.  What are they supposed to use to predict that?

And it can be a problem on medium to small forums depending on the number of members, notifications selected and topics, and the server eMail limit.  I recommend you test this on a server and force the email over the limit once or twice and watch what happens with the users there before implementing this as a user selection.  I can tell you it created total chaos on one of my forums, so much so that I no longer allow ANY notifications on that board because I don't want to spend all night managing it.  You end up with many users reporting they didn't get notified of X, and while you're looking into it, they get the notification for something they already read and wonder why.

If you want to make something user selectable, make it a single notification per user that tells them new posts were made today... go there and read them.  The forum already uses "new" indicators that tell you where the new posts are, so why duplicate all of that for the convenience of some users.  Isn't that what Mods were made for?








« Last Edit: Jun 5th, 2009 at 4:47pm by OH Eng »  

 
Back to top
 
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Add once-daily "digest" option for notifications
Reply #8 - Jun 5th, 2009 at 5:32pm
Post Tools
Unilat wrote on Jun 5th, 2009 at 2:06pm:
Just use AJAX to send a request out to the server. The server will run it and no client will have to wait for results.

That's also a very good idea, and something to consider.  But, just having the email process run after the user request is finished would accomplish much the same thing, so if AJAX proves more difficult to implement, there is still a solid solution.

OH Eng wrote on Jun 5th, 2009 at 4:40pm:
Admins have no idea how many emails/hr will ever be sent.

No, but the software does.  When I said "selectable" I mean that the Admin can either enable or disable load management.  If load management is enabled, then the script should limit the number of emails per batch.  There is no need for the admin to know anything.

OH Eng wrote on Jun 5th, 2009 at 4:40pm:
depending on the number of members, notifications selected and topics, and the server eMail limit.

The notifications selected and number of topics is irrelevant for digest mode - that's the whole point of digest mode.  You are guaranteed to send no more than one email per member per day, and that's if every member selects to be notified.  The email limit could be an issue, but that's the whole point of load management - the script can be configured to send no more than X emails per Y minutes, whatever your ISP's limit is.

OH Eng wrote on Jun 5th, 2009 at 4:40pm:
I recommend you test this on a server and force the email over the limit once or twice

None of my servers have an email limit, so I can't test that.  However, under the current system of "sending out as many emails as needed with no limit," I have no problems with my rather mediocre server in terms of CPU load.  Using digest mode would only reduce the number of emails sent.

OH Eng wrote on Jun 5th, 2009 at 4:40pm:
Isn't that what Mods were made for?

Dude, of course Mods are there for that, but this is a forum for suggesting new features, and hence I'm suggesting new features.  At least a few other devs think it's a good idea, so I'm clearly not alone in this.

I'm not trying to imply that YaBB is substandard, nor am I trying to get features implemented that only I would like.  I am suggesting features because that's what the forum is here for, and if other people agree that those features would be useful, then maybe there's something to it.  If nobody else wants the feature, well, the thread can always just languish.

What's the problem?
« Last Edit: Jun 5th, 2009 at 5:34pm by cepheid »  
Back to top
WWW  
IP Logged
 
OH Eng
Past Team Members
Documentation Team
Offline



Posts: 4,026
Location: Pensacola, Florida USA
Re: Add once-daily "digest" option for notifications
Reply #9 - Jun 6th, 2009 at 4:37am
Post Tools
There is no problem, I'm providing feedback to what you posted... isn't that the point when you post a suggestion?  I'm pointing out that making something selectable by Admin whether to auto-manage or not means that some user will select "not" and if so, runs the risk of overloading his forum.

All I am suggesting to you is to go test it under the most UNfavorable conditions it could possibly run in and see how well it works.  I have... it caused what I consider problems, thus I ended up turning the entire feature OFF.  So I'm going to offer that advice when I see someone suggest a way for a user to cripple his own forum without knowing what he's doing, and it has nothing to do with how useful a function it may be.



 

  

 
Back to top
 
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Add once-daily "digest" option for notifications
Reply #10 - Jun 6th, 2009 at 5:03am
Post Tools
OH Eng wrote on Jun 6th, 2009 at 4:37am:
So I'm going to offer that advice when I see someone suggest a way for a user to cripple his own forum without knowing what he's doing

Sure, I understand... although the current implementation allows just that already.  In fact, it's "worse" because there's no limit on the number of emails that can be sent per day this way.  Implementing a digest mode, even without any sort of load management, is guaranteed to be better than the current method.
  
Back to top
WWW  
IP Logged
 
OH Eng
Past Team Members
Documentation Team
Offline



Posts: 4,026
Location: Pensacola, Florida USA
Re: Add once-daily "digest" option for notifications
Reply #11 - Jun 6th, 2009 at 4:41pm
Post Tools
I have no problem whatsoever with a digest mode.  I have a problem with not defaulting it to a managed mode to keep the forum safe from overloading itself, or providing a on/off function for management that warns the user of possible problems.  That's the only issue I have with all of it.
  

 
Back to top
 
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Add once-daily "digest" option for notifications
Reply #12 - Jun 7th, 2009 at 12:05am
Post Tools
OH Eng wrote on Jun 6th, 2009 at 4:41pm:
I have a problem with not defaulting it to a managed mode to keep the forum safe from overloading itself, or providing a on/off function for management that warns the user of possible problems.

We can default it to a managed mode, sure.  As for providing a warning to the user, the current system doesn't warn the user... though I guess you don't like that, either! Smiley  All those suggestions are good.
  
Back to top
WWW  
IP Logged
 
Matt Siegman
YaBB Legends (Inactive)
*
Offline



Posts: 3,380
Location: Wichita, KS
Re: Add once-daily "digest" option for notifications
Reply #13 - Jun 8th, 2009 at 5:01pm
Post Tools
The only really good way to do this is to kick off a daily cron job. I'd struggle to say even half of our users have cron access.

It's a great idea. Maybe as a seperate package or a big mod this could be done, but I hesitate to support something that will not work for most of our userbase. (I really do like daily digest emails. Trust me, if there was a good way to do this, we would.)
  

-- Matt Siegman 8) Wish List
Back to top
 
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Add once-daily "digest" option for notifications
Reply #14 - Jun 8th, 2009 at 7:29pm
Post Tools
Matt Siegman wrote on Jun 8th, 2009 at 5:01pm:
The only really good way to do this is to kick off a daily cron job.

That's the preferred way and the only way to guarantee that emails get sent out on time, but why do you think having the script handle it is not a reasonable alternative?  The time check adds essentially no processing; handling the email is the only load-intensive procedure and that can happen via a spawned process so it never interferes with the user.

The only downside is that the time the emails are sent depends on the script access - a very unpopular forum with only a few hits a days wouldn't work well with this method, but then, if it's only getting a few hits per day, the digest is irrelevant.  The digest is most useful for medium to large forums with moderate to heavy activity, and those will get sufficient hits (either from users or from search engines) to trigger the digest email processing relatively close to the specified time.
  
Back to top
WWW  
IP Logged
 
Page Index Toggle Pages: [1] 2 
Topic Tools
 
  « Board Index ‹ Board  ^Top