Zikula Community

• dreamingmonkey
  
  

  • Rank: Softmore
  • Registered: Nov 25, 2007
  • Last visit: May 30, 2010
  • Posts: 166

  Posted: Jul 09, 2008 - 04:55 AM

  Does anyone know of a module that builds complex category filters? I've been working with the News module and it is only set up to filter for one property at a time (prop == cat).
  
  I want to be able to do something like:
  
  prop1 == cat21
  AND
  (prop2 == cat15 or prop2==cat35)
  
  I know I could gather the results for each property / category pair individually and then compare the results, coming up with the combinations I want, but that's very inefficient, especially if I were to attempt something like this:
  
  How about setting multiple categories per property for an individual item. Say I wanted a local news story to show up in the town to which it applies and every town within a 10-mile radius. Assuming each town is its own category (in my case, it is), I'd like to be able to set the value of the HomeTown property to an array containing the IDs (in my case, zip codes) of all the applicable categories. Then have the filter check to see if the current home town's zip code (stored in a session variable) matches any value in an that array. Otherwise, I'd be left having to set the HomeTown property to a single category and then build a filter for each of the following and combine the output:
  
  prop2 == 61723 OR prop2 == 61754 OR prop2 == 61724 OR prop2 == 61775, etc, etc, etc. Fairly easy to do, really, but again, it doesnt' seem very efficient.
  
  Am I dreaming way too big for what the Categories system is designed to deal with? If not, has someone done something that I can peak at that might set me on the right path?
  
  --
  Help Now! Fast and affordable help for do-it-yourself webmasters from Wicked Viral :: Chicago's Only Web Development Firm Specializing in Social Networking Integration

  • 
  •  
• nestormateo
  
  

  • Rank: Team Member
  • Registered: Sep 06, 2006
  • Last visit: May 09, 2010
  • Posts: 2446

  Posted: Jul 09, 2008 - 07:13 AM

  MMmmmmmm
  the categories system definitely can help you to achieve this, but i see two issues:
  
  1. Ouch again! I didn't considered the AND between the two properties, i guess we need a new __META__.field for this "exclusive" cases...
  
  2. You may set the home town's Zip codes as attributes of their categories. Then, design a plugin i guess, with an algorithm to set up the range of Zip codes (?) around the current home town, and search directly in the 'objectdata_attributes' table (WHERE oba_attribute_name = 'hometown_zipcode' AND oba_value >= $min AND oba_value
  
  --
  - Mateo T. -
  Mis principios... son mis fines

  • 
  •  
• dreamingmonkey
  
  

  • Rank: Softmore
  • Registered: Nov 25, 2007
  • Last visit: May 30, 2010
  • Posts: 166

  Posted: Jul 09, 2008 - 12:51 PM

  Thanks for the encouragement, Mateo. As for #2, after we worked out that last problem the other night, I emptied out my categories and imported a CSV file I generated from a database of all towns in the U.S. so that the category ID for each town is the zip code. I know it's cheating, but it was far easier than setting up a category for each zip code (32,000+). And since the module that actually runs the whole show contains pre-defined lists of zip codes within 10, 25, and 100 miles of each zip code, I have the cats array already built for every query (provided I want results within 10, 25, or 100 miles). So now it's just a simple matter of re-working the way the News module (the only module I'm using extensively that I didn't write myself) builds the category filter. Shouldn't be too bad.
  
  As for the AND question... I'd love to help you out on that one, but I'm really a bit of a hacker. If I get something working (I've been playing with a couple ideas since I originally posted this), I'll throw it up somewhere as a proof of concept, but I can pretty much guarantee that you'll want to run my code through the washing machine before you try to add it to anything. I'm the guy you call when you have 30 different data files all in different formats and you need a little bit from each and you need it by 8:00 tomorrow morning. I'll get it done, but it ain't gonna be pretty.
  
  By the way, since you're apparently one of the key people on the development of the categories system, I should let you know that the system really does sort of crap out at 32,000+ categories. Most of the problems are actually on the UI end of things. There is a parameter that can be sent to one of the functions that generates subcategories lists that stops it from returning leaf nodes, but no way to set that parameter from the UI functions. It's only set and passed among internal functions of the classes. If I set the default so that leaf nodes aren't returned, then I have to change all my other leaf nodes (those that aren't zip codes) to non-leaf. Not ideal, and sort of a headache. So I've simply hacked it up so that it doesn't return leaf nodes under the "grandparent" of the zip codes (tree goes HomeTowns > State > Zip). And I've had to strip the category filter out of the admin view page of the News module because even allocating 128 MB of memory to PHP, I'd exceed my allocated limit (pretty bad considering the table itself is somewhere around 6 MB).
  
  I'm sure nobody probably envisioned that large a set of categories, but I've had to do quite a bit of hacking at the code (and my server configuration) to keep it breathing. If you (or anyone else on the team) wants to have something to stress-test the system against, I'll be happy to provide you with the csv file I imported into the categories table (or just an SQL dump of the table itself if that's easier) so that you can see where the choking points are. It's certainly not "top secret" data. It's just stuff I pulled from the census bureau's web site and ran through a half dozen of the ugliest shell and PHP scripts you've ever seen.
  
  --
  Help Now! Fast and affordable help for do-it-yourself webmasters from Wicked Viral :: Chicago's Only Web Development Firm Specializing in Social Networking Integration

  • 
  •  
• dreamingmonkey
  
  

  • Rank: Softmore
  • Registered: Nov 25, 2007
  • Last visit: May 30, 2010
  • Posts: 166

  Posted: Jul 14, 2008 - 04:45 PM

  Update :: I've solved it...sort of.
  
  The AND problem is tricky. I would have had to re-write the entire _generateCategoryFilter function, which didn't seem like a good idea. So here's what I did:
  
  1. I added a parameter called $useAnd to selectObjectArrayFilter that I can pass from the calling API function. It defaults to false.
  
  2. I pass that parameter from selectObjectArrayFilter to generateCategoryFilterWhere.
  
  3. Inside generateCategoryFilterWhere, I use the value of $useAnd to determine what do to next. If it's set to false, I populate $idlist as usual (using _generateCategeoryFilter). If it's set to true, I populate it with a new function that I called _generateCategoryFilterWithAnd, which performs a query on each prop/cat pair, reading the results into an array. I use array_intersect to create an array containing only those IDs that are returned by every prop/cat query and return that.
  
  It's not an elegant solution, but it works. So, if anyone else needs to do something like this, that's a basic road map. If anyone would like the code, send me a PM. I'm not going to submit it as a patch because:
  
  1. I think it's pretty ugly.
  2. I don't feel like I have a full grasp on the overall philosophy of the Category system, and I doubt this solution is likely to resemble the final solution to the problem.
  
  But, it does stop the bleeding for me.
  
  --
  Help Now! Fast and affordable help for do-it-yourself webmasters from Wicked Viral :: Chicago's Only Web Development Firm Specializing in Social Networking Integration

  • 
  •  
• nestormateo
  
  

  • Rank: Team Member
  • Registered: Sep 06, 2006
  • Last visit: May 09, 2010
  • Posts: 2446

  Posted: Jul 15, 2008 - 05:20 AM

  Hey dreaming monkey
  soon when i have some time i'll add the __META__.useand field (or something like that ) ti the $catFilter and submit the patch for the 1.0.2 i think.
  
  CYA
  
  --
  - Mateo T. -
  Mis principios... son mis fines

  • 
  •  
• dreamingmonkey
  
  

  • Rank: Softmore
  • Registered: Nov 25, 2007
  • Last visit: May 30, 2010
  • Posts: 166

  Posted: Jul 15, 2008 - 05:37 AM

  Thanks. I changed it so that now I'm using __META__.useand inside the category filter rather than passing an additional parameter. I don't know why I didn't do that to begin with, especially after you mentioned it above.
  
  Be warned, though, I wasn't able to get the SQL to work when I changed OR to AND in the appropriate place in _generateCategeoryFilter. It kept returning 0 results. But it was my third 20-hour day in the past 5 days, so that might have been the problem. That's why I created a new function.
  
  --
  Help Now! Fast and affordable help for do-it-yourself webmasters from Wicked Viral :: Chicago's Only Web Development Firm Specializing in Social Networking Integration

  • 
  •  
•  
•  

Users on-line

• michiel

This list is based on users active over the last 60 minutes.