Fixes on billing and added support for async backend actions

This commit is contained in:
Marc Aymerich 2015-05-14 13:28:54 +00:00
parent a78c7e2769
commit cf2215f604
18 changed files with 329 additions and 108 deletions

31
TODO.md
View File

@ -174,7 +174,7 @@ require_once(/etc/moodles/.$moodle_host.config.php);``` moodle/drupl
* allow empty metric pack for default rates? changes on rating algo * allow empty metric pack for default rates? changes on rating algo
# don't produce lines with cost == 0 or quantity 0 ? maybe minimal quantity for billing? like 0.1 ? or minimal price? per line or per bill? # don't produce lines with cost == 0 or quantity 0 ? maybe minimal quantity for billing? like 0.1 ? or minimal price? per line or per bill?
# lines too long on invoice, double lines or cut, and make margin wider # lines too long on invoice, double lines or cut
* payment methods icons * payment methods icons
* use server.name | server.address on python backends, like gitlab instead of settings? * use server.name | server.address on python backends, like gitlab instead of settings?
@ -183,11 +183,11 @@ require_once(/etc/moodles/.$moodle_host.config.php);``` moodle/drupl
* update service orders on a celery task? because it take alot * update service orders on a celery task? because it take alot
# FIXME do more test, make sure billed until doesn't get uodated whhen services are billed with les metric, and don't upgrade billed_until when undoing under this circumstances # FIXME do more test, make sure billed until doesn't get uodated whhen services are billed with les metric, and don't upgrade billed_until when undoing under this circumstances
* line 513: change threshold and one time service metric change should update last value if not billed, only record for recurring invoicing. postpay services should store the last metric for pricing period. # * line 513: change threshold and one time service metric change should update last value if not billed, only record for recurring invoicing. postpay services should store the last metric for pricing period.
* add ini, end dates on bill lines and breakup quanity into size(defaut:1) and metric # * add ini, end dates on bill lines and breakup quanity into size(defaut:1) and metric
* threshold for significative metric accountancy on services.handler # * threshold for significative metric accountancy on services.handler
* http://orchestra.pangea.org/admin/orders/order/6418/ # * http://orchestra.pangea.org/admin/orders/order/6418/
* http://orchestra.pangea.org/admin/orders/order/6495/bill_selected_orders/ # * http://orchestra.pangea.org/admin/orders/order/6495/bill_selected_orders/
* move normurlpath to orchestra.utils from websites.utils * move normurlpath to orchestra.utils from websites.utils
@ -254,8 +254,6 @@ https://code.djangoproject.com/ticket/24576
* move all tests to django-orchestra/tests * move all tests to django-orchestra/tests
* *natural keys: those fields that uniquely identify a service, list.name, website.name, webapp.name+account, make sure rest api can not edit thos things * *natural keys: those fields that uniquely identify a service, list.name, website.name, webapp.name+account, make sure rest api can not edit thos things
# migrations accounts, bill, orders, auth -> migrate the rest (contacts lambda error)
* MultiCHoiceField proper serialization * MultiCHoiceField proper serialization
@ -275,19 +273,14 @@ https://code.djangoproject.com/ticket/24576
# bill.totals make it 100% computed? # bill.totals make it 100% computed?
* joomla: wget https://github.com/joomla/joomla-cms/releases/download/3.4.1/Joomla_3.4.1-Stable-Full_Package.tar.gz -O - | tar xvfz - * joomla: wget https://github.com/joomla/joomla-cms/releases/download/3.4.1/Joomla_3.4.1-Stable-Full_Package.tar.gz -O - | tar xvfz -
# bill confirmation: show total
# Amend lines??? # Amend lines???
# orders currency setting # orders currency setting
# Determine the difference between data serializer used for validation and used for the rest API! # Determine the difference between data serializer used for validation and used for the rest API!
# Make PluginApiView that fills metadata and other stuff like modeladmin plugin support # Make PluginApiView that fills metadata and other stuff like modeladmin plugin support
# custom validation for settings
# TODO orchestra related services code reload: celery/uwsgi reloading find aonther way without root and implement reload # TODO orchestra related services code reload: celery/uwsgi reloading find aonther way without root and implement reload
# insert settings on dashboard dynamically
# convert all complex settings to string
# size monitor of @002 @003 database names # size monitor of @002 @003 database names
# password validation cracklib on change password form=????? # password validation cracklib on change password form=?????
# reset setting button # reset setting button
@ -353,18 +346,20 @@ make django admin taskstate uncollapse fucking traceback, ( if exists ?)
resorce monitoring more efficient, less mem an better queries for calc current data resorce monitoring more efficient, less mem an better queries for calc current data
# test best_price rating method
# bill this https://orchestra.pangea.org/admin/orders/order/8236/ should be already billed, <= vs < # bill this https://orchestra.pangea.org/admin/orders/order/8236/ should be already billed, <= vs <
# Convert rating method from function to PluginClass # Convert rating method from function to PluginClass
# Tests can not run because django.db.utils.ProgrammingError: relation "accounts_account" does not exist # Tests can not run because django.db.utils.ProgrammingError: relation "accounts_account" does not exist
# autoresponses on mailboxes, not addresses or remove them # autoresponses on mailboxes, not addresses or remove them
# Async specific backend actions? systemusers.set_permission # ACL don't give exec permissions to files!
# force save and continue on routes (and others?)
# gevent for python3 # gevent for python3
apt-get install cython3 apt-get install cython3
export CYTHON='cython3' export CYTHON='cython3'
pip3 install https://github.com/fantix/gevent/archive/master.zip pip3 install https://github.com/fantix/gevent/archive/master.zip
# SIgnal handler for notify workers to reload stuff, like resource sync: https://docs.python.org/2/library/signal.html
# INVOICE fucking Id based on order ID or what?

View File

@ -113,6 +113,7 @@ class ChangeAddFieldsMixin(object):
add_form = None add_form = None
add_prepopulated_fields = {} add_prepopulated_fields = {}
change_readonly_fields = () change_readonly_fields = ()
change_form = None
add_inlines = None add_inlines = None
def get_prepopulated_fields(self, request, obj=None): def get_prepopulated_fields(self, request, obj=None):
@ -151,8 +152,12 @@ class ChangeAddFieldsMixin(object):
def get_form(self, request, obj=None, **kwargs): def get_form(self, request, obj=None, **kwargs):
""" Use special form during user creation """ """ Use special form during user creation """
defaults = {} defaults = {}
if obj is None and self.add_form: if obj is None:
if self.add_form:
defaults['form'] = self.add_form defaults['form'] = self.add_form
else:
if self.change_form:
defaults['form'] = self.change_form
defaults.update(kwargs) defaults.update(kwargs)
return super(ChangeAddFieldsMixin, self).get_form(request, obj, **defaults) return super(ChangeAddFieldsMixin, self).get_form(request, obj, **defaults)

View File

@ -22,7 +22,7 @@ class Mailbox(models.Model):
related_name='mailboxes') related_name='mailboxes')
filtering = models.CharField(max_length=16, filtering = models.CharField(max_length=16,
default=settings.MAILBOXES_MAILBOX_DEFAULT_FILTERING, default=settings.MAILBOXES_MAILBOX_DEFAULT_FILTERING,
choices=[(k, v[0]) for k,v in settings.MAILBOXES_MAILBOX_FILTERINGS.items()]) choices=[(k, v[0]) for k,v in sorted(settings.MAILBOXES_MAILBOX_FILTERINGS.items())])
custom_filtering = models.TextField(_("filtering"), blank=True, custom_filtering = models.TextField(_("filtering"), blank=True,
validators=[validators.validate_sieve], validators=[validators.validate_sieve],
help_text=_("Arbitrary email filtering in sieve language. " help_text=_("Arbitrary email filtering in sieve language. "

View File

@ -82,18 +82,30 @@ MAILBOXES_MAILBOX_FILTERINGS = Setting('MAILBOXES_MAILBOX_FILTERINGS',
{ {
# value: (verbose_name, filter) # value: (verbose_name, filter)
'DISABLE': (_("Disable"), ''), 'DISABLE': (_("Disable"), ''),
'REJECT': (mark_safe_lazy(_("Reject spam (Score&ge;9)")), textwrap.dedent(""" 'REJECT': (mark_safe_lazy(_("Reject spam (Score&ge;9)")), textwrap.dedent("""\
require ["fileinto","regex","envelope","vacation","reject","relational","comparator-i;ascii-numeric"]; require ["fileinto","regex","envelope","vacation","reject","relational","comparator-i;ascii-numeric"];
if header :value "ge" :comparator "i;ascii-numeric" "X-Spam-Score" "9" { if header :value "ge" :comparator "i;ascii-numeric" "X-Spam-Score" "9" {
discard; discard;
stop; stop;
}""")), }""")),
'REDIRECT': (mark_safe_lazy(_("Archive spam (Score&ge;9)")), textwrap.dedent(""" 'REJECT5': (mark_safe_lazy(_("Reject spam (Score&ge;5)")), textwrap.dedent("""\
require ["fileinto","regex","envelope","vacation","reject","relational","comparator-i;ascii-numeric"];
if header :value "ge" :comparator "i;ascii-numeric" "X-Spam-Score" "5" {
discard;
stop;
}""")),
'REDIRECT': (mark_safe_lazy(_("Archive spam (Score&ge;9)")), textwrap.dedent("""\
require ["fileinto","regex","envelope","vacation","reject","relational","comparator-i;ascii-numeric"]; require ["fileinto","regex","envelope","vacation","reject","relational","comparator-i;ascii-numeric"];
if header :value "ge" :comparator "i;ascii-numeric" "X-Spam-Score" "9" { if header :value "ge" :comparator "i;ascii-numeric" "X-Spam-Score" "9" {
fileinto "Spam"; fileinto "Spam";
stop; stop;
}""")), }""")),
'REDIRECT5': (mark_safe_lazy(_("Archive spam (Score&ge;5)")), textwrap.dedent("""\
require ["fileinto","regex","envelope","vacation","reject","relational","comparator-i;ascii-numeric"];
if header :value "ge" :comparator "i;ascii-numeric" "X-Spam-Score" "5" {
fileinto "Spam";
stop;
}""")),
'CUSTOM': (_("Custom filtering"), lambda mailbox: mailbox.custom_filtering), 'CUSTOM': (_("Custom filtering"), lambda mailbox: mailbox.custom_filtering),
} }
) )

View File

@ -3,6 +3,7 @@ from django.utils.html import escape
from django.utils.safestring import mark_safe from django.utils.safestring import mark_safe
from django.utils.translation import ugettext_lazy as _ from django.utils.translation import ugettext_lazy as _
from orchestra.admin import ExtendedModelAdmin
from orchestra.admin.utils import admin_link, admin_date, admin_colored, display_mono from orchestra.admin.utils import admin_link, admin_date, admin_colored, display_mono
from . import settings, helpers from . import settings, helpers
@ -23,13 +24,28 @@ STATE_COLORS = {
} }
class RouteAdmin(admin.ModelAdmin): from django import forms
from orchestra.forms.widgets import SpanWidget
from orchestra.forms.widgets import paddingCheckboxSelectMultiple
class RouteForm(forms.ModelForm):
def __init__(self, *args, **kwargs):
super(RouteForm, self).__init__(*args, **kwargs)
if self.instance:
self.fields['backend'].widget = SpanWidget()
self.fields['backend'].required = False
self.fields['async_actions'].widget = paddingCheckboxSelectMultiple(45)
self.fields['async_actions'].choices = ((action, action) for action in self.instance.backend_class.actions)
class RouteAdmin(ExtendedModelAdmin):
list_display = ( list_display = (
'backend', 'host', 'match', 'display_model', 'display_actions', 'async', 'is_active' 'backend', 'host', 'match', 'display_model', 'display_actions', 'async', 'is_active'
) )
list_editable = ('host', 'match', 'async', 'is_active') list_editable = ('host', 'match', 'async', 'is_active')
list_filter = ('host', 'is_active', 'async', 'backend') list_filter = ('host', 'is_active', 'async', 'backend')
ordering = ('backend',) ordering = ('backend',)
add_fields = ('backend', 'host', 'match', 'async', 'is_active')
change_form = RouteForm
BACKEND_HELP_TEXT = helpers.get_backends_help_text(ServiceBackend.get_backends()) BACKEND_HELP_TEXT = helpers.get_backends_help_text(ServiceBackend.get_backends())
DEFAULT_MATCH = { DEFAULT_MATCH = {

View File

@ -62,7 +62,9 @@ def generate(operations):
if operation.routes is None: if operation.routes is None:
operation.routes = router.get_routes(operation, cache=cache) operation.routes = router.get_routes(operation, cache=cache)
for route in operation.routes: for route in operation.routes:
key = (route, operation.backend) # TODO key by action.async
async_action = route.action_is_async(operation.action)
key = (route, operation.backend, async_action)
if key not in scripts: if key not in scripts:
backend, operations = (operation.backend(), [operation]) backend, operations = (operation.backend(), [operation])
scripts[key] = (backend, operations) scripts[key] = (backend, operations)
@ -111,13 +113,13 @@ def execute(scripts, serialize=False, async=None):
threads_to_join = [] threads_to_join = []
logs = [] logs = []
for key, value in scripts.items(): for key, value in scripts.items():
route, __ = key route, __, async_action = key
backend, operations = value backend, operations = value
args = (route.host,) args = (route.host,)
if async is None: if async is None:
is_async = not serialize and route.async is_async = not serialize and (route.async or async_action)
else: else:
is_async = not serialize and async is_async = not serialize and (async or async_action)
kwargs = { kwargs = {
'async': is_async, 'async': is_async,
} }

View File

@ -0,0 +1,20 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
import orchestra.models.fields
class Migration(migrations.Migration):
dependencies = [
('orchestration', '0003_auto_20150512_1512'),
]
operations = [
migrations.AddField(
model_name='route',
name='async_actions',
field=orchestra.models.fields.MultiSelectField(blank=True, max_length=256),
),
]

View File

@ -8,7 +8,7 @@ from django.utils.module_loading import autodiscover_modules
from django.utils.translation import ugettext_lazy as _ from django.utils.translation import ugettext_lazy as _
from orchestra.core.validators import validate_ip_address, ValidationError from orchestra.core.validators import validate_ip_address, ValidationError
from orchestra.models.fields import NullableCharField from orchestra.models.fields import NullableCharField, MultiSelectField
#from orchestra.utils.apps import autodiscover #from orchestra.utils.apps import autodiscover
from . import settings from . import settings
@ -157,10 +157,13 @@ class Route(models.Model):
async = models.BooleanField(default=False, async = models.BooleanField(default=False,
help_text=_("Whether or not block the request/response cycle waitting this backend to " help_text=_("Whether or not block the request/response cycle waitting this backend to "
"finish its execution. Usually you want slave servers to run asynchronously.")) "finish its execution. Usually you want slave servers to run asynchronously."))
async_actions = MultiSelectField(max_length=256, blank=True,
help_text=_("Specify individual actions to be executed asynchronoulsy."))
# method = models.CharField(_("method"), max_lenght=32, choices=method_choices, # method = models.CharField(_("method"), max_lenght=32, choices=method_choices,
# default=MethodBackend.get_default()) # default=MethodBackend.get_default())
is_active = models.BooleanField(_("active"), default=True) is_active = models.BooleanField(_("active"), default=True)
class Meta: class Meta:
unique_together = ('backend', 'host') unique_together = ('backend', 'host')
@ -210,6 +213,9 @@ class Route(models.Model):
name = type(exception).__name__ name = type(exception).__name__
raise ValidationError(': '.join((name, exception))) raise ValidationError(': '.join((name, exception)))
def action_is_async(self, action):
return action in self.async_actions
def matches(self, instance): def matches(self, instance):
safe_locals = { safe_locals = {
'instance': instance, 'instance': instance,

View File

@ -42,6 +42,7 @@ class BillSelectedOrders(object):
proforma=form.cleaned_data['proforma'], proforma=form.cleaned_data['proforma'],
new_open=form.cleaned_data['new_open'], new_open=form.cleaned_data['new_open'],
) )
print(self.options)
if int(request.POST.get('step')) != 3: if int(request.POST.get('step')) != 3:
return self.select_related(request) return self.select_related(request)
else: else:

View File

@ -8,6 +8,7 @@ from orchestra.contrib.bills.models import Invoice, Fee, ProForma
class BillsBackend(object): class BillsBackend(object):
def create_bills(self, account, lines, **options): def create_bills(self, account, lines, **options):
bill = None bill = None
ant_bill = None
bills = [] bills = []
create_new = options.get('new_open', False) create_new = options.get('new_open', False)
proforma = options.get('proforma', False) proforma = options.get('proforma', False)
@ -17,17 +18,23 @@ class BillsBackend(object):
continue continue
service = line.order.service service = line.order.service
# Create bill if needed # Create bill if needed
if bill is None or service.is_fee:
if proforma: if proforma:
if ant_bill is None:
if create_new: if create_new:
bill = ProForma.objects.create(account=account) bill = ProForma.objects.create(account=account)
else: else:
bill = ProForma.objects.filter(account=account, is_open=True).last() bill = ProForma.objects.filter(account=account, is_open=True).last()
if not bill: if not bill:
bill = ProForma.objects.create(account=account, is_open=True) bill = ProForma.objects.create(account=account, is_open=True)
bills.append(bill)
else:
bill = ant_bill
ant_bill = bill
elif service.is_fee: elif service.is_fee:
bill = Fee.objects.create(account=account) bill = Fee.objects.create(account=account)
bills.append(bill)
else: else:
if ant_bill is None:
if create_new: if create_new:
bill = Invoice.objects.create(account=account) bill = Invoice.objects.create(account=account)
else: else:
@ -35,6 +42,9 @@ class BillsBackend(object):
if not bill: if not bill:
bill = Invoice.objects.create(account=account, is_open=True) bill = Invoice.objects.create(account=account, is_open=True)
bills.append(bill) bills.append(bill)
else:
bill = ant_bill
ant_bill = bill
# Create bill line # Create bill line
billine = bill.lines.create( billine = bill.lines.create(
rate=service.nominal_price, rate=service.nominal_price,

View File

@ -50,7 +50,7 @@ class BillSelectRelatedForm(AdminFormMixin, forms.Form):
billing_point = forms.DateField(widget=forms.HiddenInput()) billing_point = forms.DateField(widget=forms.HiddenInput())
fixed_point = forms.BooleanField(widget=forms.HiddenInput(), required=False) fixed_point = forms.BooleanField(widget=forms.HiddenInput(), required=False)
proforma = forms.BooleanField(widget=forms.HiddenInput(), required=False) proforma = forms.BooleanField(widget=forms.HiddenInput(), required=False)
create_new_open = forms.BooleanField(widget=forms.HiddenInput(), required=False) new_open = forms.BooleanField(widget=forms.HiddenInput(), required=False)
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
super(BillSelectRelatedForm, self).__init__(*args, **kwargs) super(BillSelectRelatedForm, self).__init__(*args, **kwargs)
@ -64,4 +64,4 @@ class BillSelectConfirmationForm(AdminFormMixin, forms.Form):
billing_point = forms.DateField(widget=forms.HiddenInput()) billing_point = forms.DateField(widget=forms.HiddenInput())
fixed_point = forms.BooleanField(widget=forms.HiddenInput(), required=False) fixed_point = forms.BooleanField(widget=forms.HiddenInput(), required=False)
proforma = forms.BooleanField(widget=forms.HiddenInput(), required=False) proforma = forms.BooleanField(widget=forms.HiddenInput(), required=False)
create_new_open = forms.BooleanField(widget=forms.HiddenInput(), required=False) new_open = forms.BooleanField(widget=forms.HiddenInput(), required=False)

View File

@ -64,7 +64,7 @@ class RateQuerySet(models.QuerySet):
return self.filter( return self.filter(
Q(plan__is_default=True) | Q(plan__is_default=True) |
Q(plan__contracts__account=account) Q(plan__contracts__account=account)
).order_by('plan', 'quantity').select_related('plan') ).order_by('plan', 'quantity').select_related('plan', 'service')
class Rate(models.Model): class Rate(models.Model):

View File

@ -44,24 +44,24 @@ def _compute_steps(rates, metric):
return value, steps return value, steps
def _prepend_missing(rates): def _standardize(rates):
""" """
Support for incomplete rates Support for incomplete rates
When first rate (quantity=5, price=10) defaults to nominal_price When first rate (quantity=5, price=10) defaults to nominal_price
""" """
if rates: std_rates = []
first = rates[0] minimal = rates[0].quantity
if first.quantity == 0: for rate in rates:
first.quantity = 1 if rate.quantity == 0:
elif first.quantity > 1: rate.quantity = 1
if not isinstance(rates, list): elif rate.quantity == minimal and rate.quantity > 1:
rates = list(rates) service = rate.service
service = first.service rate_class = type(rate)
rate_class = type(first) std_rates.append(
rates.insert(0, rate_class(service=service, plan=rate.plan, quantity=1, price=service.nominal_price)
rate_class(service=service, plan=first.plan, quantity=1, price=service.nominal_price)
) )
return rates std_rates.append(rate)
return std_rates
def step_price(rates, metric): def step_price(rates, metric):
@ -71,7 +71,7 @@ def step_price(rates, metric):
group = [] group = []
minimal = (sys.maxsize, []) minimal = (sys.maxsize, [])
for plan, rates in rates.group_by('plan').items(): for plan, rates in rates.group_by('plan').items():
rates = _prepend_missing(rates) rates = _standardize(rates)
value, steps = _compute_steps(rates, metric) value, steps = _compute_steps(rates, metric)
if plan.is_combinable: if plan.is_combinable:
group.append(steps) group.append(steps)
@ -122,7 +122,7 @@ def step_price(rates, metric):
minimal = min(minimal, (value, result), key=lambda v: v[0]) minimal = min(minimal, (value, result), key=lambda v: v[0])
return minimal[1] return minimal[1]
step_price.verbose_name = _("Step price") step_price.verbose_name = _("Step price")
step_price.help_text = _("All rates with a quantity lower than the metric are applied. " step_price.help_text = _("All rates with a quantity lower or equal than the metric are applied. "
"Nominal price will be used when initial block is missing.") "Nominal price will be used when initial block is missing.")
@ -132,7 +132,7 @@ def match_price(rates, metric):
candidates = [] candidates = []
selected = False selected = False
prev = None prev = None
rates = _prepend_missing(rates.distinct()) rates = _standardize(rates.distinct())
for rate in rates: for rate in rates:
if prev: if prev:
if prev.plan != rate.plan: if prev.plan != rate.plan:
@ -163,25 +163,42 @@ def best_price(rates, metric):
raise ValueError("rates queryset should be ordered by 'plan' and 'quantity'") raise ValueError("rates queryset should be ordered by 'plan' and 'quantity'")
candidates = [] candidates = []
for plan, rates in rates.group_by('plan').items(): for plan, rates in rates.group_by('plan').items():
rates = _prepend_missing(rates) rates = _standardize(rates)
plan_candidates = [] plan_candidates = []
for rate in rates: for rate in rates:
if rate.quantity > metric: if rate.quantity > metric:
break break
if plan_candidates: if plan_candidates:
plan_candidates[-1].barrier = rate.quantity ant = plan_candidates[-1]
if ant.price == rate.price:
# Multiple plans support
ant.fold += 1
else:
ant.quantity = rate.quantity-1
plan_candidates.append(AttrDict( plan_candidates.append(AttrDict(
price=rate.price, price=rate.price,
barrier=metric, quantity=metric,
fold=1,
))
else:
plan_candidates.append(AttrDict(
price=rate.price,
quantity=metric,
fold=1,
)) ))
candidates.extend(plan_candidates) candidates.extend(plan_candidates)
results = [] results = []
accumulated = 0 accumulated = 0
for candidate in sorted(candidates, key=lambda c: c.price): for candidate in sorted(candidates, key=lambda c: c.price):
if accumulated+candidate.barrier > metric: if candidate.quantity < accumulated:
# Out of barrier
continue
candidate.quantity *= candidate.fold
if accumulated+candidate.quantity > metric:
quantity = metric - accumulated quantity = metric - accumulated
else: else:
quantity = candidate.barrier quantity = candidate.quantity
accumulated += quantity
if quantity: if quantity:
if results and results[-1].price == candidate.price: if results and results[-1].price == candidate.price:
results[-1].quantity += quantity results[-1].quantity += quantity
@ -192,4 +209,4 @@ def best_price(rates, metric):
})) }))
return results return results
best_price.verbose_name = _("Best price") best_price.verbose_name = _("Best price")
best_price.help_text = _("Produces the best possible price given all active rating lines.") best_price.help_text = _("Produces the best possible price given all active rating lines (those with quantity lower or equal to the metric).")

View File

@ -214,6 +214,7 @@ class Service(models.Model):
return decimal.Decimal(str(accumulated)) return decimal.Decimal(str(accumulated))
ant_counter = counter ant_counter = counter
accumulated += rate['price'] * rate['quantity'] accumulated += rate['price'] * rate['quantity']
raise RuntimeError("Rating algorithm bad result")
else: else:
if metric < position: if metric < position:
raise ValueError("Metric can not be less than the position.") raise ValueError("Metric can not be less than the position.")
@ -221,6 +222,7 @@ class Service(models.Model):
counter += rate['quantity'] counter += rate['quantity']
if counter >= position: if counter >= position:
return decimal.Decimal(str(rate['price'])) return decimal.Decimal(str(rate['price']))
raise RuntimeError("Rating algorithm bad result")
def get_rates(self, account, cache=True): def get_rates(self, account, cache=True):
# rates are cached per account # rates are cached per account

View File

@ -32,8 +32,8 @@ class HandlerTests(BaseTestCase):
'orchestra.contrib.systemusers', 'orchestra.contrib.systemusers',
) )
def create_ftp_service(self): def create_ftp_service(self, **kwargs):
service = Service.objects.create( default = dict(
description="FTP Account", description="FTP Account",
content_type=ContentType.objects.get_for_model(SystemUser), content_type=ContentType.objects.get_for_model(SystemUser),
match='not systemuser.is_main', match='not systemuser.is_main',
@ -46,10 +46,18 @@ class HandlerTests(BaseTestCase):
on_cancel=Service.DISCOUNT, on_cancel=Service.DISCOUNT,
payment_style=Service.PREPAY, payment_style=Service.PREPAY,
tax=0, tax=0,
nominal_price=10, nominal_price=10
) )
default.update(kwargs)
service = Service.objects.create(**default)
return service return service
def validate_results(self, rates, results):
self.assertEqual(len(rates), len(results))
for rate, result in zip(rates, results):
self.assertEqual(rate['price'], result.price)
self.assertEqual(rate['quantity'], result.quantity)
def test_get_chunks(self): def test_get_chunks(self):
service = self.create_ftp_service() service = self.create_ftp_service()
handler = service.handler handler = service.handler
@ -239,9 +247,7 @@ class HandlerTests(BaseTestCase):
'quantity': 21 'quantity': 21
} }
] ]
for rate, result in zip(rates, results): self.validate_results(rates, results)
self.assertEqual(rate['price'], result.price)
self.assertEqual(rate['quantity'], result.quantity)
dupeplan = Plan.objects.create( dupeplan = Plan.objects.create(
name='DUPE', allow_multiple=True, is_combinable=True) name='DUPE', allow_multiple=True, is_combinable=True)
@ -249,9 +255,7 @@ class HandlerTests(BaseTestCase):
service.rates.create(plan=dupeplan, quantity=3, price=9) service.rates.create(plan=dupeplan, quantity=3, price=9)
results = service.get_rates(account, cache=False) results = service.get_rates(account, cache=False)
results = service.rate_method(results, 30) results = service.rate_method(results, 30)
for rate, result in zip(rates, results): self.validate_results(rates, results)
self.assertEqual(rate['price'], result.price)
self.assertEqual(rate['quantity'], result.quantity)
account.plans.create(plan=dupeplan) account.plans.create(plan=dupeplan)
results = service.get_rates(account, cache=False) results = service.get_rates(account, cache=False)
@ -261,9 +265,7 @@ class HandlerTests(BaseTestCase):
{'price': decimal.Decimal('9.00'), 'quantity': 5}, {'price': decimal.Decimal('9.00'), 'quantity': 5},
{'price': decimal.Decimal('1.00'), 'quantity': 21}, {'price': decimal.Decimal('1.00'), 'quantity': 21},
] ]
for rate, result in zip(rates, results): self.validate_results(rates, results)
self.assertEqual(rate['price'], result.price)
self.assertEqual(rate['quantity'], result.quantity)
hyperplan = Plan.objects.create( hyperplan = Plan.objects.create(
name='HYPER', allow_multiple=False, is_combinable=False) name='HYPER', allow_multiple=False, is_combinable=False)
@ -276,9 +278,8 @@ class HandlerTests(BaseTestCase):
{'price': decimal.Decimal('0.00'), 'quantity': 19}, {'price': decimal.Decimal('0.00'), 'quantity': 19},
{'price': decimal.Decimal('5.00'), 'quantity': 11} {'price': decimal.Decimal('5.00'), 'quantity': 11}
] ]
for rate, result in zip(rates, results): self.validate_results(rates, results)
self.assertEqual(rate['price'], result.price)
self.assertEqual(rate['quantity'], result.quantity)
hyperplan.is_combinable = True hyperplan.is_combinable = True
hyperplan.save() hyperplan.save()
results = service.get_rates(account, cache=False) results = service.get_rates(account, cache=False)
@ -287,9 +288,7 @@ class HandlerTests(BaseTestCase):
{'price': decimal.Decimal('0.00'), 'quantity': 23}, {'price': decimal.Decimal('0.00'), 'quantity': 23},
{'price': decimal.Decimal('1.00'), 'quantity': 7} {'price': decimal.Decimal('1.00'), 'quantity': 7}
] ]
for rate, result in zip(rates, results): self.validate_results(rates, results)
self.assertEqual(rate['price'], result.price)
self.assertEqual(rate['quantity'], result.quantity)
service.rate_algorithm = 'orchestra.contrib.plans.ratings.match_price' service.rate_algorithm = 'orchestra.contrib.plans.ratings.match_price'
service.save() service.save()
@ -335,9 +334,7 @@ class HandlerTests(BaseTestCase):
'quantity': 21 'quantity': 21
} }
] ]
for rate, result in zip(rates, results): self.validate_results(rates, results)
self.assertEqual(rate['price'], result.price)
self.assertEqual(rate['quantity'], result.quantity)
def test_zero_rates(self): def test_zero_rates(self):
service = self.create_ftp_service() service = self.create_ftp_service()
@ -357,9 +354,7 @@ class HandlerTests(BaseTestCase):
{'price': decimal.Decimal('9.00'), 'quantity': 6}, {'price': decimal.Decimal('9.00'), 'quantity': 6},
{'price': decimal.Decimal('1.00'), 'quantity': 21} {'price': decimal.Decimal('1.00'), 'quantity': 21}
] ]
for rate, result in zip(rates, results): self.validate_results(rates, results)
self.assertEqual(rate['price'], result.price)
self.assertEqual(rate['quantity'], result.quantity)
def test_rates_allow_multiple(self): def test_rates_allow_multiple(self):
service = self.create_ftp_service() service = self.create_ftp_service()
@ -375,9 +370,7 @@ class HandlerTests(BaseTestCase):
{'price': decimal.Decimal('0.00'), 'quantity': 2}, {'price': decimal.Decimal('0.00'), 'quantity': 2},
{'price': decimal.Decimal('9.00'), 'quantity': 28}, {'price': decimal.Decimal('9.00'), 'quantity': 28},
] ]
for rate, result in zip(rates, results): self.validate_results(rates, results)
self.assertEqual(rate['price'], result.price)
self.assertEqual(rate['quantity'], result.quantity)
account.plans.create(plan=dupeplan) account.plans.create(plan=dupeplan)
results = service.get_rates(account, cache=False) results = service.get_rates(account, cache=False)
@ -386,9 +379,7 @@ class HandlerTests(BaseTestCase):
{'price': decimal.Decimal('0.00'), 'quantity': 4}, {'price': decimal.Decimal('0.00'), 'quantity': 4},
{'price': decimal.Decimal('9.00'), 'quantity': 26}, {'price': decimal.Decimal('9.00'), 'quantity': 26},
] ]
for rate, result in zip(rates, results): self.validate_results(rates, results)
self.assertEqual(rate['price'], result.price)
self.assertEqual(rate['quantity'], result.quantity)
account.plans.create(plan=dupeplan) account.plans.create(plan=dupeplan)
results = service.get_rates(account, cache=False) results = service.get_rates(account, cache=False)
@ -397,6 +388,150 @@ class HandlerTests(BaseTestCase):
{'price': decimal.Decimal('0.00'), 'quantity': 6}, {'price': decimal.Decimal('0.00'), 'quantity': 6},
{'price': decimal.Decimal('9.00'), 'quantity': 24}, {'price': decimal.Decimal('9.00'), 'quantity': 24},
] ]
for rate, result in zip(rates, results): self.validate_results(rates, results)
self.assertEqual(rate['price'], result.price)
self.assertEqual(rate['quantity'], result.quantity) def test_best_price(self):
service = self.create_ftp_service(rate_algorithm='orchestra.contrib.plans.ratings.best_price')
account = self.create_account()
dupeplan = Plan.objects.create(name='DUPE')
account.plans.create(plan=dupeplan)
service.rates.create(plan=dupeplan, quantity=0, price=0)
service.rates.create(plan=dupeplan, quantity=2, price=9)
service.rates.create(plan=dupeplan, quantity=3, price=8)
service.rates.create(plan=dupeplan, quantity=4, price=7)
service.rates.create(plan=dupeplan, quantity=5, price=10)
service.rates.create(plan=dupeplan, quantity=10, price=5)
raw_rates = service.get_rates(account, cache=False)
results = service.rate_method(raw_rates, 2)
rates = [
{
'price': decimal.Decimal('0.00'),
'quantity': 1
},
{
'price': decimal.Decimal('9.00'),
'quantity': 1
},
]
self.validate_results(rates, results)
results = service.rate_method(raw_rates, 3)
rates = [
{
'price': decimal.Decimal('0.00'),
'quantity': 1
},
{
'price': decimal.Decimal('8.00'),
'quantity': 2
},
]
self.validate_results(rates, results)
results = service.rate_method(raw_rates, 5)
rates = [
{
'price': decimal.Decimal('0.00'),
'quantity': 1
},
{
'price': decimal.Decimal('7.00'),
'quantity': 4
},
]
self.validate_results(rates, results)
results = service.rate_method(raw_rates, 9)
rates = [
{
'price': decimal.Decimal('0.00'),
'quantity': 1
},
{
'price': decimal.Decimal('7.00'),
'quantity': 4
},
{
'price': decimal.Decimal('10.00'),
'quantity': 4
},
]
self.validate_results(rates, results)
results = service.rate_method(raw_rates, 10)
rates = [
{
'price': decimal.Decimal('0.00'),
'quantity': 1
},
{
'price': decimal.Decimal('5.00'),
'quantity': 9
},
]
self.validate_results(rates, results)
def test_best_price_multiple(self):
service = self.create_ftp_service(rate_algorithm='orchestra.contrib.plans.ratings.best_price')
account = self.create_account()
dupeplan = Plan.objects.create(name='DUPE')
account.plans.create(plan=dupeplan)
account.plans.create(plan=dupeplan)
service.rates.create(plan=dupeplan, quantity=0, price=0)
service.rates.create(plan=dupeplan, quantity=2, price=9)
service.rates.create(plan=dupeplan, quantity=3, price=8)
service.rates.create(plan=dupeplan, quantity=4, price=7)
service.rates.create(plan=dupeplan, quantity=5, price=10)
service.rates.create(plan=dupeplan, quantity=10, price=5)
raw_rates = service.get_rates(account, cache=False)
results = service.rate_method(raw_rates, 3)
rates = [
{
'price': decimal.Decimal('0.00'),
'quantity': 2
},
{
'price': decimal.Decimal('8.00'),
'quantity': 1
},
]
self.validate_results(rates, results)
results = service.rate_method(raw_rates, 10)
rates = [
{
'price': decimal.Decimal('0.00'),
'quantity': 2
},
{
'price': decimal.Decimal('5.00'),
'quantity': 8
},
]
self.validate_results(rates, results)
account.plans.create(plan=dupeplan)
raw_rates = service.get_rates(account, cache=False)
results = service.rate_method(raw_rates, 3)
rates = [
{
'price': decimal.Decimal('0.00'),
'quantity': 3
},
]
self.validate_results(rates, results)
results = service.rate_method(raw_rates, 10)
rates = [
{
'price': decimal.Decimal('0.00'),
'quantity': 3
},
{
'price': decimal.Decimal('5.00'),
'quantity': 7
},
]
self.validate_results(rates, results)

View File

@ -29,9 +29,9 @@ class WebAppServiceMixin(object):
if context['under_construction_path']: if context['under_construction_path']:
self.append(textwrap.dedent("""\ self.append(textwrap.dedent("""\
if [[ $CREATED == 1 && ! $(ls -A %(app_path)s) ]]; then if [[ $CREATED == 1 && ! $(ls -A %(app_path)s) ]]; then
# Async wait for other backends to do their thing or cp under construction # Async wait 2 more seconds for other backends to lock app_path or cp under construction
nohup bash -c ' nohup bash -c '
sleep 10 sleep 2
if [[ ! $(ls -A %(app_path)s) ]]; then if [[ ! $(ls -A %(app_path)s) ]]; then
cp -r %(under_construction_path)s %(app_path)s cp -r %(under_construction_path)s %(app_path)s
chown -R %(user)s:%(group)s %(app_path)s chown -R %(user)s:%(group)s %(app_path)s

View File

@ -43,7 +43,8 @@ class WordPressBackend(WebAppServiceMixin, ServiceController):
die("App directory not empty."); die("App directory not empty.");
} }
shell_exec("mkdir -p %(app_path)s shell_exec("mkdir -p %(app_path)s
rm -f %(app_path)s/index.html # Prevent other backends from writting here
touch %(app_path)s/.lock
filename=\\$(wget https://wordpress.org/latest.tar.gz --server-response --spider --no-check-certificate 2>&1 | grep filename | cut -d'=' -f2) filename=\\$(wget https://wordpress.org/latest.tar.gz --server-response --spider --no-check-certificate 2>&1 | grep filename | cut -d'=' -f2)
mkdir -p %(cms_cache_dir)s mkdir -p %(cms_cache_dir)s
if [ \\$(basename \\$(readlink %(cms_cache_dir)s/wordpress) 2> /dev/null ) != \\$filename ]; then if [ \\$(basename \\$(readlink %(cms_cache_dir)s/wordpress) 2> /dev/null ) != \\$filename ]; then
@ -54,7 +55,9 @@ class WordPressBackend(WebAppServiceMixin, ServiceController):
tar -xzvf %(cms_cache_dir)s/wordpress -C %(app_path)s --strip-components=1 tar -xzvf %(cms_cache_dir)s/wordpress -C %(app_path)s --strip-components=1
fi fi
mkdir %(app_path)s/wp-content/uploads mkdir %(app_path)s/wp-content/uploads
chmod 750 %(app_path)s/wp-content/uploads"); chmod 750 %(app_path)s/wp-content/uploads
rm %(app_path)s/.lock
");
$config_file = file('%(app_path)s/' . 'wp-config-sample.php'); $config_file = file('%(app_path)s/' . 'wp-config-sample.php');
$secret_keys = file_get_contents('https://api.wordpress.org/secret-key/1.1/salt/'); $secret_keys = file_get_contents('https://api.wordpress.org/secret-key/1.1/salt/');

View File

@ -27,10 +27,6 @@ class MultiSelectField(models.CharField, metaclass=models.SubfieldBase):
def to_python(self, value): def to_python(self, value):
if value: if value:
# if isinstance(value, tuple) and value[0].startswith('('):
# # Workaround unknown bug on default model values
# # [u"('SUPPORT'", u" 'ADMIN'", u" 'BILLING'", u" 'TECH'", u" 'ADDS'", u" 'EMERGENCY')"]
# value = list(eval(', '.join(value)))
if isinstance(value, str): if isinstance(value, str):
return value.split(',') return value.split(',')
return value return value
@ -44,10 +40,11 @@ class MultiSelectField(models.CharField, metaclass=models.SubfieldBase):
setattr(cls, 'get_%s_display' % self.name, func) setattr(cls, 'get_%s_display' % self.name, func)
def validate(self, value, model_instance): def validate(self, value, model_instance):
if self.choices:
arr_choices = self.get_choices_selected(self.get_choices_default()) arr_choices = self.get_choices_selected(self.get_choices_default())
for opt_select in value: for opt_select in value:
if (opt_select not in arr_choices): if (opt_select not in arr_choices):
msg = self.error_messages['invalid_choice'] % value msg = self.error_messages['invalid_choice'] % {'value': opt_select}
raise exceptions.ValidationError(msg) raise exceptions.ValidationError(msg)
def get_choices_selected(self, arr_choices=''): def get_choices_selected(self, arr_choices=''):